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(54) IMAGE DATA INTERPOLATING DEVICE, IMAGE DATA INTERPOLATING METHOD, AND 
MEDIUM ON WHICH IMAGE DATA INTERPOLATING PROGRAM IS RECORDED 

(57) When a plurality of types of iniage data are 
superposed and written onto a color Information virtual 
drawing screen, attribute information of each pixel is 
written onto an attribute information virttal drawing 
screen so that the image data can be read out for every 
type. When results of superposition are read out from a 
virtual drawing screen for an interpolating process, the 
results are read out for every type of image data on the 
basis of the attribute information so that an optimum 
interpolating process suitable for each type is carried 
out. Since a margin of each image data is affected by 
the interpolating process, superposition after the inter- 
polating process is controlled so that the margin is ren- 
dered most suitable. As a result, optimum results of 
interpolation can be achieved even when a plurality of 
types of image data are co-existent. 
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d^ects to be proce^ed B cortained. and a medium on which a 
lOOM] To aaomplish the object, the invention of claim 1 provides an image data irtewSii a^ratS^Si 
obtains .mage data containing attribute intomation capable of distinguishing a t^ SiSge S3^.SeTS 
'^l"^!! ^ interpolating process and which con^rrses a rLdJTuS S r2dro2^tK^?i^ 

date^ characterized by an interpolating unit which distinguishes a pl^ity of image tjjes rtTe S tesSrthe 
^^^^^ and appI.«one Of a plurality of interpolating proce Jes diffen-ng «S eveVoneTSe i^e^P^ 
prtSSi * synthesizing unH which synthesizes the pixels interpolated by 3,e different 1!^^ 

^ *® '^^"S?" ^ ""s*"*^** as described above, the image data is obtained and enlartied bv the 

inteipdabng process The image data contains attribute information capable 6f recognfeSSTSL d irSfor aTrC 

E^^;"e irSJS""" r '"terpol^nit disS^shSimS^^^'rS tS pTe2 

b^ed on the attrtoute information and applies one of a pluiality of interpolating procises differiiw fSWery one S me 

i^JJinn la^^^'T*^'^-^!"^^^ '"'^"^^ ^^"^ « most suitable pixel interpolating process differs 

according to the types. Accordingly, the image data containing the several types of in^ges is recwnizL foleSrv 

lE nnJ?? **^!! '® representing a pattern constituted by dot-matrix pixels and should not be 

^Tn^n ^ ^ ^ P^^ot°9^aPh or chamcters. Further, the iLge data itoelf m^beVse?<?3ote 

drawing an image or fonts comprising vector information. e« w arawing commanas t or 

ITvSli, the^^iS SiT"*^"^- ^'^^^ differentiating properties of images and is held so as to be read 

JS*e?i^ Tn t^e '''^ ^^^^ ^ ^^^tten onto a 

^Jfof S TiT^nS^fJ^^'l^^ * '^^ ««s case, the image data interpolating appa- 
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[001 5] The types in the image data are recognizable for every pixel. Various techniques to render the types recog^ 
nizabie can be enployed. For example, an attribute area may separately be provided so that types of individual data are 
written as attributes onto the virtual region. Consequently, the type of each pixel can be found when the attribute area 
is referred to. In this case, writing may be performed by the virtual drawing unit 

5 [001 6] The virtual region may be prepared for every type of image data and have a layer structure with a text screen 
and a natural screen. The enlarging process may be can^ied out while the image data is being input from the layer struc- 
ture by an application. Furthermore, a part of the image data recognizable of the attn^bute of the image data may be 
read out for every type, and the renr^ning part of the image data may be undistinguishable. 
[0017] The readout unit reads out the image data of every pixel for every type. For example, when the type can be 

10 determined from the attribute area, the readout unit selects the image data to be read out while referring to the attribute 
area. 

[0018] Further, since a two-dimensional processing is performed In the interpolating process, the Image data needs 
to be input accordingly For this purpose, when the image data is read out from the virtual region, a plurality of lines of 
the image data may be input for the interpolating process. As a result, a two-dimensional interpolating process can be 
15 realized. 

[001 9] Various types of interpolating process may be employed. For example, the interpolating process by the cubic 
method is suitalsle for a natural image though unsuitable for a business graph. On the other hand, the nearest methtod 
is suitable for a non-natural image such as the business graph though unsuitable for the natural image. Whether the 
image data is a natural image or a non-natural image is a kind of characteristic of the image. The interpolating process 
20 is selected according to such a characteristic. Further, the imerpolating process for the natural image can sometimes 
be changed depending upon an ot^ect. For example, the interpolating process may be changed between a daylight 
photograph and a night photograph. More specifically, the characteristic of the image may only affect the result of inter- 
potation when the interpolating process is changed. 

[0020] As another example, the interpolating unit may be provided with pattern data corresponding to presence or 
25 absence of pixel information in a predetermined area and interpolation pixel information with a predetermined interpo- 
lating scale factor corresponding to each pattern data. The interpolating unit may input a pixel of the corresponding area 
from the virtual region and compare the pixel with the pattern data, so that the interpolating process is performed on 
the basis of the interpolation pixel information corresponding to the matched pattern data. 

[0021] In the case of the above*described construction, the interpolating process is carried out by the pattern 

30 matching with respect to a predetermined small area. More specifically, the pattern data corresponding to presence or 
absence of the pixel information corresponding to the small area is prepared, and a pixel in the corresponding area is 
read out from the virtual region and compared with the pattern data. The interpolating process is performed on the basis 
of the interpolation pixel information of a predetermined interpolating scale factor con-esponding to the pattern data 
which matches the pattern data. Accordingly, an expected noise pixel can be prepared as the pattern data and the inter- 

35 polation pixel information in which the noise pixel is deleted according to ihe pattem data. 

[0022] In the aforesaid pattern matching, reneviring all the comparison data is troublesome when an object area is 
moved so that a new pattern matching is performed, tn this case, the pattern data may be a rectangular area containing 
pixels whose number con^esponds to a concurrently processible data width, and a new sequence of pixels is incorpo- 
rated into the conparison data in a moving direction of the rectangular area by a first-tn first-out process so tiiat the 

40 matching with the pattem data is continued. 

[0023] In the above-descnlded arrangement the pattern matching can be carried out by one operational processing 
per area in the case of a rectangular area containing pixels whose numtser corresponds to a concurrently processible 
data width. Further, when the object area is moved so that a new pattern matching is carried out, not all the comparison 
data need be renewed, and a new sequence of pixels is incorporated into the connparison data in a moving direction by 

45 the f irst-in first-out process. More specifically, the comparison with pattern data of 16 pixels is performed in the pattem 
matching of pixels whose mrvber is 4x4. when a square area is nKsved by one pixel, information atx)ut three rows of 
pixels does not sut^stantiafly change. Information about presence or absence of one row of four pixels at the forward 
side relative to the moving cirection is incorporated into the conparison data, and information about presence or 
absence of one row of four pixels at the backward side is out of target Accordingly, the f irst-in first-out is carried out with 

so respect to the four pixels, so that not all the comparison data need be renewed. Consequently, the pattern matching can 
be carried out easily and efficiently. 

[0024] The determination cannot be made on the t^asis of only the preserve or absence of pixels when the pattern 
matching is applied to a color image. Accordingly, pattern data needs to be prepared for every color. tHJt this is unreal- 
istic. On the other hand, the interpolated image data corresponding to the pattem data may be arranged to include color 
£5 arrangement information of every color in the comparison data. 

[0025] In the above<iescribed arrangement, the pixels are matched with the comparison data representative of the 
presence or absence of pixels. Since the interpolated image infonnation referred to when the pixels have been matched 
with the comparison data includes the color arrangemait information, the interpolation of the color image by the pattem 
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.rXt: ^S^STrS^hl^"'"' ^"^^'-"•'y- ♦''^ "-se a- be interpo- 

[0026] The synthesizing unit synthesizes image data for which the pixels thereof have been interpolated. In this 

^ when interpolating unit is arranged to temporally hold the results of inteipolating process for every image data in 

anottier region, the image data held in the region is superposed in a predetermined sequence. Atternatively. the results 
<rf interpolation may be written onto a predetermined output region with the interpolating process being oamed out in 
the predetermined sequence. m 
[0027] AWiough the synthesizing unit thus synthesizes the image data in the predetermined sequence, the super- 
position can be performed more satisfactorily according to the character of the interpolating process. As an examoie 

10 the invention claimed in claim 3 is constmcted so that in the image interpolating apparatus of daim 1 or 2 the synthe-' 
toTS ile Ss ^ * "^^^^ processing unit which adjusts superposition of margins of the image data aft'er interpda- 

[0028] In the invention of claim 3 thus constmcted, the margin processing unit of the symhesizing unit adjusts the 
superposifion of margins of the image data after the image interpolation. 

w [0029] Since the interpolating process generates new pixels and there are different techniques in the interpolating 
process, a configuration of the margin varies when different interpolating processes are applied. For example when 
there are an interpolating process resulting in a large variation in the marginal configuration and another interpdating 
process maintaining an original marginal configuration, the superposition is preferably carried out on the basis of the 
marginal configuration in the latter. In this sense, the margin processing unH acQusts the superposition. 

so [0030] The adjustment by the margin processing unit may be changed according to the interpolating process As 
an sample, the margin processing unit may superpose a plurality of types of image data for which the pixels thereof 
have been interpolated, in a sequence predetermined according to the interpolating process. 
[0M1] In the above-described anangement. the margin is adjusted by superposing the image data in the sequence 
predetermined according to the interpolating process. In the aforesaid example, there are an interpolating process 

25 resulting in a large variation in the marginal configuration and another interpolating process maintaining an original mar- 
ginal configuration. In this case, the former is first written onto a predetenrtned output region and thereafter the latter 
is ovenvritten such that the marginal configuration of the latter is used. 

[0032] As another exanple. the margin processing unit m^ first write onto the output region the image data corre- 
sponding to an interpolating process in which the margin is expanded. 
30 [0033] In the above-described arrangement when there is image data in which the margin is expanded as the 
result of the interpolating process, the margin processing unit iirs\ writes that image data to be interpolatal by the inter- 
polating process onto the output region. The margin is narrowed or expanded depending tpon the interpolating proc- 
ess. The margin intrudes into an adjacent region when expanded. Accordingly, the image data is first written onto the 
output region and the marginal portion is ovenwritten so that a percentage of the portion intruding into the adjacent 
region is substantially reduced, whereupon the marginal configuration is maintained. 

[0034] Furthemiore. the margin processing unit may write tt>e image data for which an interpolating process in 
which the marginal configuration is smoothed is can^ied out. later than the image data for which another interpolatina 
process Is carried out 

[0035] When there are the Interpolating process in which the marginal configuration is smoothed and another inter- 
polating process, the interpolating process in which the marginal configuration is smoottied can maintain the marginal 
configuration. Accordingly, when the image data corresponding to the Interpolating process in which the marginal con- 
figuration is smoothed is first written onto the output region, the marginal configuration which should not be maintained 
IS maintained, which is inconvenient. Accordingly, the image data con-esponding to the interpolating process in which 
the marginal configuration is smoothed is later written onto the output region. 

[0036] Whether the marginal configuration is maintained depends upon a purpose. For example, the margin is eas- 
ily smoothed in the pattern matcWng. There is a case where the marginal configuration is maintained when such 
smoothing is canied out On the otiier hand. aHhough shagginess becomes more conspicuous as an interpolating scale 
factor IS inaeased, as in tiie nearest method, the marginal configuration can be maintained. 
[0037] Furthermore, the invention claimed in daim 4 is constmcted so that in the image data interpolating appara- 
tus of daim 3. when the readout unit reads out the image data, the margin processing unit causes the readout unit to 
read out the image data witti a margin enlarged, thereby superposing ttie image data on the image data interpolated by 
the interpolating unit on the basis of the image data with the enlarged.margin. 

[0038] . In the invention of daim 4 thus constitirted. when the readout unit reads out the image date, the margin 
processing unit causes the readout unit to read out the image date with a margin enlarged. The image date is then 
superposed on the image data interpolated by the interpdating unit on the basis of the image date with the enlarged 

margin. ^ 

[0039] The resultant interpolation image data is expanded when ttie original image data is previously expanded. 
Then, even when the margin of the resultant interpolation image data adjacent to the expanded image data does not 
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match that of the expanded image data, that margin iG reserved ae a foundation and the missing of pixels is prevented. 
[0040] Further, the invention claimed in daim 5 is constructed so that in the image data interpolating apparatus of 
claim 4, the margin processing unit enlarges the nrargin of the image data with respect to an interpolating process in 
which Information outside the margin Is drawn in. 

5 [0041 ] In the invention of claim 5 thus constructed, when there is an interpolating process in which information out- 
side the margin is drawn in. the margin is enlarged as described above and the Interpolating process is then performed. 
The post-interpolation image data is written onto the output region etc. When the interpolating process draws in infor> 
mation outside the margin* information is cfiluted since information of a part containing no pixels is drawn in, which influ- 
ences the margin. On the other hand, when the margin Is previously enlarged, the influenced margin is concealed under 

10 the margin of the adjacent image data, whereupon the influence is eliminated. 

[00421 There are various types of image data, which can mainly be classified into metacommand image data and 
non-metaconrtmand image data. As an exanpte surtat>le for such a case, the invention claimed in claim 6 is constructed 
so that in the image data interpoiating apparatus of any one of claims 1 to 5. said plurality of types of image data having 
different image types Include Image data corresponding to a metacommand and other Image data, which further com- 

15 prises a non-metacommand pixel interpolating unit which enlarges a marginal region when the pixel corresponding to 
the image data other than the metacommand and performs an interpolating process so that a predetermined interpo- 
lation scale fector is obtained, and a metacommand pixel interpolating unit which generates an Interpolated pixel so that 
the Interpolated pixel corresponds to the original metacommand when reading out the pixel con^espdrtding to the met- 
acomnruuid and performing an interpolating process so that the interpolating scale factor is obtained, wherein the syn- 

so thesizing unit synthesizes a result of interpolatbn by the non-metacommand pixel interpolating unit and a result of 
interpolation by the metacommand pixel interpolating unit, the synthesizing unit preferring the result of interpolation by 
the metacommand pixel interpolating unit with respect to the superposed portion. 

[0043] In the invention of claim 6 thus constructed, the non-metacommand pixel interpolating unit reads out from 
the vlrtial region etc. the pixels con^esponding to the image data oth^ than the metacommand and performs an inter- 
ns polating process so that a predetermined interpdation scale factor is obtain&j. In this case, the interpolating process is 
performed with the marginal region being enlarged. Accordingly an interpolated image enlarged relative to the original 
region is obtained. On the other hand, the metacommand pixel interpolating unit also reads out from the virtual region 
eta the pixels con^esponding to the metacommand and performs an interpolating process so that the predetemiined 
interpolation scale factor is obtained. In the interpolating process, the interpolated pixels are generated so ^ to corre- 
30 spond to the original metacommarvj. The synthesizing unit synthesizes a result of interpolation by the non-metacom- 
mand pixel interpolating unit and a result of interpolation by the metacommand pixel interpolating unit. The synthesizing 
unit prefers the result of interpolation by the metacommand pixel interpolating mrt with respect to the superposed por- 
tion. 

[0044] More specif ically, the metacommand image is a mass of dot-matrix pixels in the virtual region etc. The met- 

35 acommand image has a smoothed contour corresponding to the original metacommand in the interpolating process, 
but the contour is necessarily changed from that before the interpolating process. When the contour is thus changed, 
a part of the metacommand image may be superposed on the acQacent image other than the metacommand or a gap 
may be formed. On the other hand, the image other than the metacommand is generated so as to be larger than the 
original and accordingly, there is less possibility of occurrence of the gap and the image of the metacommand is pre- 

40 ferred in the superposed portion. Consequently, a smoothed contour remains. 

[0045] The metacommand used here means a vectorial representation of shape. Accordingly, a drawing application 
would translate the command, drawing a graphic, and the image quality is not deteriorated eve if enlargement or reduc- 
tion are repeated. On the other hand, tnforrrBtion at>out each pixel is given when the image other than the metacom- 
mand is drawn. The information is lost when the image is reduced and cannot be returned even when the image is 

45 enlarged. In this sense, the metacommand is used for characters as well as for images. 

[0046] The characteristic of the metacommarxi having such a property cannot always be maintained in the 
processing by a computer. Accordingly, when the metacommand is represented as a mass of pixels at one time, it 
needs to be subjected to the same processing as applied to the other image thereafter. However, when whether an 
image has been drawn by the metacommand, the interpolating technique needs to be changed in the interpolating 

so process in which the image is enlarged. For example, it is considered that an image by the metacommand should gen- 
erally be interpolated with a margin being smoothed. On the other hand, whether the margin should be smoothed can- 
not be determined unconditionally regarding the other image. Accordingly when the metacommand and non- 
metacommarxJ inages are interpolated in the manners different from eadi other, mEirginal configurations may differ 
after the interpolating processes. This is the background of the present invention. 

55 [0047] The virtual drawing unit carries out drawing on the virtual region on the basis of the image data correspond- 
ing to the metacommand and the other image data. The virtual drawing unit is capable of recognizing the image data 
corresponding to the metacommand and the other image data from each other. Various techniques rendering the rec- 
ognition possible may be employed. For example, an attribute area may separately be provided so that types of the indi- 
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^SS!.?!?hin I* °' *e '"dividual data may be provided wHh respective attributes. 

Further, wheii the number of colors Mfi be reduced, a certain bit can be applied to the attniw^ 
thSno ^ "°"-^a"'""«nd pixel interpolating unH reads out from the virtual region the pixels corresponding to 
the image data other than the metacommand, carrying out the interpolating process. F=or example, when the type of the 
.rmge data can be determined from the attribute area, the non-metacomn4Sd pixel interpolTg inH rea*3,e pbc 
els con-espondingto the image data other than the metacommand while making reference to the attribute area to select 
the image data. The non-metacommand pixel interpolating unit further interpolates the pixels by the correspondino 
interpoiattng process. 

« /^T"® interpolating manners can be enployed. For example, the imerpolating process by the cubic method 
giiS* "^^'^ ''^ ^^"^ non-natural image such as compZ 

10050] -me non-metacommand pixel interpolating unit enlarges the peripheral edge region and then carries out the 
irtteipolating proc^. Various processing manners for enlarging the peripheral edge region are employed. As an exam- 
ple^ the Invemion claimed in claim 7 Is constructed so that in the image data interpolating apparatus of daim 6. the non- 
metacommand pixel interpolating unit uses information about the pixel in the marginal region as information about a 
pixel outside the marginal region. 

■iP^^^ « invention of claim 7 thus constructed, since the peripheral edge region to be enlarged contains no 

infomiaton about pixels, information about pixels in a peripheral region is used as information about pixels in a reaion 
to be enlarged. Accordingly, the information may be copied without change or with stepwise changes. Further coding 
T^il °^ "^"9 « "^"3 fusion. Thus, an actually copying work may not be carried out and it is sufficientth^ 
the informaton is substantially usable. The size of the region to be enlarged is not limited particularly. The size of the 
region d^ends upon irregularity of the margin resulting from generation of pixels by the metacommand pixel interpo- 
lating unit. Even when the metacommand pixel has a concavity, the concavity is allowed to such an extent that the con- 
cavity does not resuft in a gap in the superposition of the image data. For exanple; when pixel infbmiation is read out 
SIffSert. """^"^ ' ^ enlarging both ends of the metacommand pixel by one pixel is 

IT® ""^co"™^ P«el interpolating unit selectively reads out the pixel corresponding to the metacom- 
mand from the virtual region. The metacommand pixel interpolating unit generates an interpolation pixel con-espondino 
to the onginal metacommand. For example, when the pixel is taken as an image, smoothing a margin or rendering a 
comer acute conesponds to this processing. " 
[00531 On the other hand, a metacommand representative of a character contains information about a figure bend- 
ing in a complicated manner in a small region. Accordingly, an extra dot tends to be generated depending upon an oper- 
ational accuracy in generation of pixels. As an example suitable for this characteristic, in the interpolating process for 
the metacommand representative of the character, a noise pixel in the generation of pixels on the basis of the meta- 
command may be deleted and the pixels may then be interpolated. 

[0054] In the above^fescribed construction, determination is made as to whether the pixel is a noise pixel when the 
pixels are generated on the basis of the metacommand representative of the character. When the pixel is a noise pixel 
the noise pixel is deleted and the pixel is then interpolated. Whether the pixel is a noise pixel can generally be deter- 
mined from a property of the character. For example, one pixel projects in spite of a straight portion. An unnaturally pro- 
jecting pixel IS in a portion where two sides intersect Or an unnaturally projecting pixel is at an end of a cun^e. That is 
the noise pixel means those which can be produced at a connection In the case where a character is represented by a 
plurality of vector data. ' 

(0055] On the other hand, a superposing unit synthesizes the result of interpolation by the non-metacommand pixel 
irterpolating unit and the result of interpolation by the metacommand pixel inteipolating unit and causes the result of 
interpolation by the metacommand pixel interpolating unft to take preference over the result of irterpolation by the non- 
metacommand pixel irterpolating unit concerning a superposed pwrtion. 

pwse] As an example of the processing in which one takes preference over the other, the invention claimed in daim 
8 IS constructed so that in the image data interpolating apparatus of any one of claims 1 to 7. the synthesizing unit syn- 
thesizes the pixels, superposing pixels in the resuH of interpolation by the metacommand pixel interpolating unit other 
manbad<ground pixels on the resuH of interpolation by the non-metacommand pixel interpolating unit 
10057] m the invention of daim 8 thus constmcted. the results of interpolating processes held in another area may 
^'^^^ '"u Predetermined sequence if the result of interpolation by the non-metacommand pixel interpolating 
unit and the result of irterpolation by the metacommand pixel irterpolating unit are preliminarily held in the another 
area. AMemahvely. the results of irterpolation may be superposed while the irterpolating processes are carried out in 
a predetermined sequence. 

S h« . JJfr^S^^.'^r^f 1^ interpolating technique should not be limHed to the substartial apparatus. It can 
easily be understood that the technique IS functioned as a method. 

[0059] The aforesaid image data interpolating apparatus may exist independentty or may be incorporated in equip- 
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ment. Thus, the scope of the invention covers various forms of implementation. Accordingly, the invention may be imple- 
mented as software or hardware. 

[0060] When the invention is implemented as sofhware for an image data interpolating apparatus, the invention 
applies equally to a medium on which the software is recoided. 

s [0061] When the invention Is Implemented as software, hardware and operating system can be employed or the 
invention may be implemented independent of them. For example, a process for inputfing image data for the interpola- 
tion can be accomplished by calling a predetennined function in an operating system or by inputting from hardware 
without calling the function. Even when the invention is actually implemented under interposition of hardware, it can be 
understood that the invention can be implemented only by a program in a stage of recording the program on a medium 

10 and circulating the medium. 

[0062] The recording medium may be a magnetic recording medium or a photomagnetic recording medium, or any 
recording medium that will be developed in the future. Further, the invention may take such a r^icated form as a pri- 
mary replicated product, secondary replicated product, etc. In addition, the invention may be supplied through use of a 
communication line. 

IS [0063] Still more, there may be provided such as anangement that some parts of the present invention are embod- 
ied in software while the other parts thereof are emtwdied in hardware. In a modified embodiment of the invention, 
some parts thereof may be formed as software recorded on a storage medium to be read into hardware as required. 
[0064] The invention will be described, merely by way of example, with reference to the accompanying drawings, in 
which: 

20 

Fig. 1 is a block diagram showing an image data interpolating apparatus of one embodiment in accordance with the 
present invention; 

Fig. 2 is a block diagram showing a concrete example of hardware for the image data interpolating apparatus; 
Fig. 3 is a schematic block diagram showing another example of apfirfication of the image data interpolating aijpa- 
25 ratus; 

Fig. 4 is a flowchart of main processing of the image data interpolating apparatus of the invention; 
Rg. 5 shows writing onto a virtual drawing screen: 

Rgs. 6A and 86B show comparison of color irrfomiation and attribute information on the virtual drawing screen; 

Fig. 7 shows the concept of the cubic method; 
30 Fig. 8 shows the concept of the nearest method; 

Fig. 9 shows a case where data of each lattice point is moved in the nearest method: 

Fig. 10 is a schematic view showing the condition before interpolation by the nearest method; 

Fig. 1 1 is a schematic view showing the condition after interpolation by the nearest method; 

Fig. 1 2 shows the concept of the bilinear method ; 
35 Fig. 1 3 is a graph showing variations in the interpolation function; 

Fig. 14 shows a character image written on the color information virtual drawing screen; 

Fig. 15 shows a case where interpolation information is obtained by pattern matching; 

Fig. 16 shows a case where interpolation information is obtained by pattern matching in differing scale factors; 

Fig. 17 shows a case where interpolation infornnation including color allocation information is obtained by pattern 
40 matching; 

ngs. 1 8A to 1 8C show a concrete data processing technique for pattern matching; 

Fig. 19 is a flowchart showing the interpolation process for a natural Image in the image data interpolating appara- 
tus of the invention; 

Fig. 20 is a flowchart showing the interpolation process for a non-natural image in the image data interpolating 
45 apparatus of the invention; 

Figs. 2 1 A and 2 1 B show a case where image data is read out for every type; 
Figs. 22A and 22B show a case where image data is read out Into a buffer for every type; 
Rgs. 23A to 23E show an inadequacy in a case where mixed image data are s^arated for the interpolation proc- 
ess; 

50 Rgs. 24A to 24D show a countermeasure against the inadequacy and its effect; 
Rg. 25 shows an effect which smoothens the margin; 

Fig. 26 is a block diagram showing the image data interpolating apparatus of another embodiment in accordance 
with the invention; 

Fig. 27 is a flowchart of main processing of the image data interpolating apparatus of the invention; 
55 Rg. 28 shows writing onto a virtual drawing screen; 

Rgs. 29A to 29D show a process of eliminating noise dot in a character by pattern matching; 
Rg. 30 is a flowchart showing the interpolatfon process for non-metacommand pixels in the image data interpolat- 
ing apparatus of the invention; 
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Fig. 31 is flowchart showing the interpolation process lor metacommand pixels in the image data interpolating 
apparatus of the invention; ^ 

Rgs. 32A to 32C show a case where image data is read out for every type and a margin extending process; 
Pigs. 33A and 33B show a case where image data is read out into a buffer for every type: and 
Figs. 34A to 34D show the concfition where the margin is extended by the margin extending process. 

embodiment of the present invention will be described with reference to the drawings 
[00661 Fig. 1 is a blod^ diagram showing an image data interpolating apparatus of one embodiment in accordance 
wtti the present invention, and Fig. 2 is a block diagram showing hardware to which the invention is applied 
[006^ in data processing in computers etc.. an image Is represented by dot-matrix pixels, and image data is com- 
pos«l of number of data each indicative of a pixel. An image should not be limited to a natural image such as photo- 
graph. A character can be an image in the sense that the character is a collection of pixels. Computer graphics and 
business graph can be images. These are commonly image data but delicately differ from each other in the properties 
and accordingly, the matching with an interpolating process differs from one another according to the properties 
[0068] In view of the differences of image properties, a virtual drawing unit C1 1 reads a plurality of types of image 
data having different image properties, renders the types distinguishable, superposes the image data in a predeter- 
mined sequence, and draws on a virtual region etc. On the other hand, a readout unit C12 and an interpolating unit C13 
read out the image data of every pixel for every attribute from the distinguishable virtual region and interpolate the pixels 
by an interpolating process according to the property of the image. A superposing unit C14 as an example of synthe- 
sizing unit writes image data interpolated according to the property of the interpolating process onto a predetermined 
output area m a superposed state. 

[00691 The embodiment employs a computer system 10 as an example of hardware realizing the image data inter- 
polating apparatus. 

I007(q Hg. 2 is a block diagram showing the computer system 2. The computer system 10 Includes as an image 
iipirt device a scanner 1 la. a digital still camera 1 lb and a video camera 1 1c all of which are connected to a computer 
1 2. Each ir«jut device generates Image data representing an image as dot-matrix pixels and is capable of oulputling the 
image data to the computer 12. The image data is represented in three primary colore of RGB in 256 gradations so that 
about 16 million and 700 thousand colors can be represented. 

[00711 Afloppy diskdrive 13a. a hard disk 13b anda CD-KM drive 13c each serving as an external auxiliary stor- 
age are connected to the computer 12. Main programs relating to the system are recorded on the hard disk 13b Other 
required programs are read from a floppy disk or CD-ROM as the occasion may demand. 

[0072] Further, a modem 14a serving as a communication device which connects the conputer 12 to an external 
network etc. is connected to the computer 1 2. The modem 1 4a is further connected to the external network via a public 
communication line through which software and data can be downloaded. Although an operator externally accesses via 
the modem 1 4a and the pidslic communication line in the embodiment, a LAN adapter may be provided for the operator 
to access the network, instead. Additionally, a keyboard 1 5a and a mouse 1 5b are provided for operation of the comou- 
ter12. f 

[00731 The computer system 1 0 further includes a display 1 7a and a color printer 1 7b each senring as an image 
output device. The display 1 7a has a display area of 800x600 pixels in the horizontal and vertical directions respectively, 
so that about 16 million and 700 thousand cotore can be displayed for every picture element as described above. How- 
ever, this is only one example of resolution. The resolution of the display 1 7a may be variable, for example it mav be 
640x480 pixels or 1024x720 pixels. ' 
[0074] On the other hand, the color printer 1 7b. which is an ink-jet printer, is capable of printing an image with dots 
on pnnting paper serving as a recording medium using four color inks of CMYK. The color printer 17b has an image 
density of 360x360 dpi or 720x720 c^ji and can thus perform a high-densHy printing. The color printer 17b has two gra- 
dations as to whether cotor ink is applied or not. 

[0075] Predetennined programs are executed in the computer 12 so that the image data input by the image data 
input device is displayed or otheravise output by the image output device. Of these programs, an operating system (OS) 
12a as a basic program runs on the computer 12. A display driver (DSP DRV) 12b and a printer driver (PRT DRV) 12c 
are incorporated in the operating system 12a. The display driver 12b instructs the display 17a to perform a displaying 
function, whereas the printer driver 1 2c instructs the color printer 1 7b to perform a printing function. These driver* 12b 
and 12c depend on the types of the display 17a and color printer 17b respectively and can accordingly be added or 
changed according to the respective types. Further, additional functions other than standard processes may be accom- 
plished depending on the respective types of the display 17a and color printer 17b. In other woids. various additional 
processes can be accomplished wHhin allowable ranges while a common processing system is maintained on the 
standard system of the operating system I2a. 

[0076] An application I2d is executed on the operating system 12a serving as the basic program. The application 
I2d has vartous processing contents. For example, it monitors operations of a keyboarel 1 5a and a mouse 15b each 
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Sr'Ji application 12d controls the external equipment so 

that the corresponding computation processing is carried out. TTie application 12d further disolavs ^b TZZ ^ 
processingonthedispiay I7aoroutputthemtolhecolorprinteri7lx ""^ °^ 

n^^nir iTiH^^^'*^ '^J^^' ^ ^ ^ sending as an image input device reads a photo- 

' nT ^ K ""T ^"^^ *^ ^ 2d tor a wo«J processor etc. carstiS read photogri^ Ke 

or business graphbased on theresults Of tabular computation as well as d^^^^ 

be output to the di^ay 1 7a or color printer 1 7b each serving as an image output device to^rdSJ^fre^^^^^^^ 

s?edt;a"re^r:;';r"'"^ 

10 [0078J When the consolidated document is output to be displayed, pixels displayed on the dlsolav 1 7a as ihev am 
cannot correspond to pixels on the color printer 1 7b. A pixel density in the case ^If^^x^^Zlnl^t^Z Zl 
applicatjon I2d and displayed on the display 1 7a is not equal to one of the color primer l T^Tc^^tZ^^T^l 

™cl?. 'T' *^ aforesaid problem, the operating system 12a detennines a reference pixel density and exe- 
cutes a resolution inversion so that the difference between the reference density and the plx2 density an aS 
dmnce » resolved. For example, when the display 17a has a resolution of 720 dpi and the operation system 12a has a 

the color pnnter 17b l«saresolution of 720 dpi under the same condition, the printer driver I2c«^^ 
conversion between them. «»wiuuwn 

[OOWa The resolution conversion is a process for increasing the number of constituent pixels of the image data and 
accordingly corresponds to an interpolating process. Each of the display driver 12b and printer driver 1 2c has a function 
of executing the interpolating process. luncuwi 

in in® «"*««'"en<- tt'e display driver 12b or the printer driver 12c writes image data on a virtual screen for 
every type of the image data so that the image data is distinguishabfe and further reads out the image data for every 
STln^J;^ virtue^ screen, wecutes ttie interpolating process in a suitable interpolating technique, superposes the 
nterpolated image data to output a fmal image to the display 17a or the color printer I7b. as will be described later In 
this sense, the display driver 12b and the printer driver 1 2c constitute the above^lescribed virtual drawing unit C1 1 the 
r^out unit C12. the interpolating unit C13 and the superposing unit C14. The display driver I2b and the printer driver 
12C are stored on the ha«l disk 13b and input to the computer 12 upon start of th^ computer system 10 £ run wJS 
put to work, these drivers recorded on a medium such as a CD-ROM or a floppy disk Jre insSed on the hart diSJ 
rS2 ' * " constitutes a medium on wWch an image data interpolating program is 

[0082] Although the image data interpolating apparatus is realized as the computer system 10 in the embodiment 

rali^ of image data having diffenng propertaes of images may be provided, instead. For example. Fig 3 shows a net- 
work computer 19a. which is connected via a public telephone line to an external wkle-area network. In such a wide- 
area networK image data having different image properties including character informatibn. photographic images eto 

tre^^^"^or"f«i^'''^; *ta'ns such image data and displays the image data on a 

television monitor 19b or output the image data to aprinter (not shown). In this case. too. the change of the image res- 
o uhon IS required, or when the operator wishes to enlarge a part of the image, the image Is interpolated with an oper- 
ation such as zooming and then displjved. t~ ^ .uiamvei 

L^n m«?*!l!!f'^'**"^. ""^-^ "^^ ^ *^ °* * *^'ay °"»P"* ^^^ce but not at the computer 

side. In the erample of a cotor pnnter. scnpt type print data is input and the interpolating process is carried out in exe- 
cution of matching with a printing resolution of the color printer. 

[0084] Fig. 4 is a flowchart of the resolution conversion canied out by the printer driver 1 2c 
[0085] At step S^^02. image data is input and sorted for superposition. More specificany. an image read from the 
appl'cafori 12d. a character input on the keyboard ISa. and a business graph generated on tabular 
calcuteton software are stuck as a single consolidated document. Superposition is required in this case. Particulariy in 
a field of DTP. an image and a character are directly superposed in most cases such that one picture is generated. A 
complicated superposition is required in this case. Although an image of a superposed lower layer is invisible, the image 
©cists on the data and accordingly, the data is again superposed by the printer driver 12c. The concept of the layer is 
ujlized whenthe images are superposed. A command of the image data is sorted so that the image data are arranged 
into upper and lower layers. The image c^a tor the lower layer Is written. 

t*^®" ^ '^'^ serving as a virtual region on the basis of the 

image data rearranged as described above. Rg. 5 typically shows writing onto the virtual drawing screen When the 
commands of the image data are sorted on the basis of the anangement of layers, drawing functions according to the 
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IrS^^^^nH^n^^.'^T'^^ ""^"^ "^^^ ^ P*^^' ^ ^^'^^ information virtual drawing 

screen and an attribute informaton virtual drawing screen each which is allocated to a memory. Three bytes corre- 
sponding to respective color components of red. green and blue for every pixel are altocated to the color information 
virtual drawing screen. An allocated memory area equals the number of pixels in the horizontal direction multiplied by 
the number of pixels in the vertical direction. The attribute intonation virtual drawing screen is provided for determining 
what each pixel is. a natural image (N). a character (C) or a business graph (B). One byte is allocated for each pixel, 
and an identification code (N. C or B) of the attribute is written onto the attribute information virtual drawing screen. Bit 
map irnage data is treated as a natural image. However, since the bit map image data is not always a natural image in 
the strict sense, the image data may be analyzed so that whether the image data is a natural Image Is determined, 
[0087] Figs. 6A and 68 show a correspondence between the color information virtual drawing screen and the 
attribute information virtual drawing screen. When one line in the horizontal direction at a reference resolution is sup- 
posed, the color and the type of each pixel Is written. Accordingly a pixel of a natural image, a character or a business 
graph can be selected from the written attribute information. 

[0088] Although the attribute information and the color Intonation are separately written onto the virtual drawing 
screen in this exanrple. a writing manner should not be limited to this. For example, four bytes may be allocated to each 
pixel, that is, one byte for the attribute intonation may be added to the color information. Further, a saeen onto which 
superposition information is written may be separated from a screen onto which color infomiation is written for every 
type, and the superposition may be earned out while the superposition infbrmatton is referred to. 
[00891 At step ST106. image data is read out for every image type from a virtual drawing screen as shown in Fig. 
5, and an optimum interpolating process according to the image type is carried out. Techniques for the interpolating 
process pr^red in the emtxxiiment will be described. 

[0090] As an interpolating process suHable for a natural image such as photograph, the interpolating process by a 
cubic method is executable. The cube method utilizes data of sixteen lattice points, that is. four lattice points surround- 
ing a point Puv to be interpolated and lattice points outside the four lattice points, as shown in Rg, 7. An equation using 
a cubic convolution function is expressed as follows: 



P=[f(yl)f(y2)f(y3)f(y4)] 
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P24 
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[0091 ] The degree of Influence according to the distance is expressed by a cubic convolution function as: 

f{t)={sin{jct))/irt 

Each of the distances x1 to x4 and y1 to y4 is obtained using an at>solute coordinate value (u. v) of the lattice point Puv 
as: 



x1=1+(u-|u|) 
x2=(u-|ul) 
x3=1.(u-|u|) 



yUl-Kv-M) 
y2=(v.|v|) 

y3=1.Hv|) 



On the atxsve assumption, the aforesaid equation is expanded as follows: 
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P*tf (yl)f (y2)f (y3)f (y4) ] 
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'Pll • f <xl) +P21 • f (x2 ) +P31- f (x3) +P41 {x4 
P12 • f (xl) +P22 • f (x2 ) +P32 - f {x3) +P42 (x4 ) 
P13 • f (xl) +P23 • f (x2 ) +P33 • f (x3) +P43 (x4 ) 
l,P14 • f (xl) +P24 • f (x2) 1-P34 • f (x3) +P44 (x4 ) 
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20 



=f (yl) (Pll-f (xl)+P21-f (x2)ip31-f (x3)+P41(x4) > 
+f (y2) (P12-£(xl)+P22-f (x2)+P32-f (x3>+P42(x4) ) 
+f (y3) {P13-f (xl)+P23-f (x2)+P33-f (x3)+P43(x4) ) 
+f (y4) {P14-f (xl)+P24-f (x2)+P34'f {x3)+P44(x4) ) 
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=Pll-f (xl)-f(yl)+P21-f(x2) •f(yl)+P31-f (x3) .f (yl)+P41(x4> -f (yl) 
■►P12-f (xD-f (y2)+P22-f (x2) •£(y2)+P32-f (x3) • f (y2)+P42 (x4) -f (y2) 
+P13-£(xl)-f (y3)+?23-f (x2) • f (y3) +P33 -f (x3) • f (y3) +P43 (x4) -f (y3) 
4-P14-f (xl)-£(y4>+P24.f (x2) -f (y4) +P34 -f (x3) • f (y4) +P44 (x4 ) -f (y4) 

The degree of Influence f(t) according to ttie distance is approximated by the fbllowing cubic equation: 

f (t)"{sin(ttt) )/jit 

{l-2|t|**2+|t|**3 :OS|t|<l 

4-8|t|+5|t|**2-|t|**3 :lS|t|<2 

? :2S|t 



45 



S^i JH^f^'" "^^"^^^ °* \rX\vBnc^ varies gradually gradually as H approaches from one lattice 

po^ to the other latbce point The cubic method has a characteristic that this variation of the influence becomes cubic 

^r^Si 1^ ^ "rterpolating process suitable for a non-natural image such as computer graphics or business graph! 

so ttie .nterpolatng process by a nearest method is executable. In the nearest method, as shown in Rg. 8. dis^e^ 
^ ^erpolafon point Puv and lour peripheral lattice points Pij. Pi+lj. Rjj+i and Pi+lj+l respectively are 
obtemed. Data of the nearest latbce point is displaced to the interpolation point Puv. This is expressed by the fbllowing 
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PuvsPij 



where i=[u^.5] and j=[v+0.5| . and each bracket indicates that an integral part is taken in the Gauss' notation 
[0094] Rg. 9 shows a case where the number of pixels is trebled both in the length and the width by the nearest 
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method. The data of the nearest one of pixels at four corners is displaced as a pixel to be interpolated. Fig. 10 shows 
the original image and Rg. 1 1 shows an image obtained by interpolating the pixels In the above-described method. The 
relation between obliquely arranged black pixels and white pixels serving as the background in the original image is 
maintained in the interpolated Image of Fig. 1 1 in which the black pixels are trebled in the number thereof and arranged 
obliquely. ^ 

[0095] In the nearest method, edges of the original image are maintained in the interpolated image without change. 
Accordingly, when the interpolated image is scaled up. jags are conspicuous although the edges are maintained. In 
other methods, on the other hand, the pixel to be interpolated is processed so as to be changed smoothly by using data 
of pixels around it As a result, although jags are less conspicuous, part of the intonation of the original data is cut oft 
such that the image is rendered edgeless. Accordingly, these other methods are unsuitable for the computer graphics 
or business graph. 

[0096] Although the above-described nearest method and cubic method are used in the embodiment, a bilinear 
interpolation method will be desaibed as another interpolation method In order that the characteristics of these meth- 
ods may be understood. 

[0097] The bilinear method is similar to the cubic method In that the influence degree varies graduaOy as it 
approaches from one lattice point to the other lattice point, as shown in Fig. 12. However, the bilinear method differs 
from the cubic method in that the variation is linear and depends upon only data of the lattice points at the opposite 
sides. More specifically, a region defined by the four lattice points Pij. Pi+1 Pij+1 and Pi+1j+1 sun-ounding the inter- 
polation point Puv is divided by the interpolation point into four subdivisions. Diagonally located data is weighted on the 
basis of area ratios among the subdivisions. This is expressed as the following equation: 

P={{i+1)-u}{(j+1)-v}Pij +{(i+1)-uHv-DPij+1 +{u-i}{(j+1)-v}Pi+1j +{u.i}{v^}Pi+1j+1 
where islu] and js[v] . 

IWQB] in each of the cubic method and the bilinear method, the influence degree varies gradually as it approaches 
from one lattice point to the other lattice point However, the variation is cubic in the cubic method, whereas the variation 
is linear in the bilinear method. This difference is large. Fig. 13 shows the tvwo-dimensional results of interpolations by 
the nearest, cubic and bilinear methods. Ihe axis of absdssas denotes a location and the axis of ordinates denotes an 
interpolation function coaesponding to the above-described influence degree according to the distance between the 
interpolation point and each lattice point. The lattice points exist at locations where t=0, t=1 and t=2. respectively. The 
Interpolation points are at locations where tsQ and 1. 

[0099] In the bilinear method, the interpolation function varies linearly between adjacent points (t=0 and 1). As a 
result, since marginal portions are smoothed, a resultant image is blurred on the screen. More specifically, when the 
marginal portions differ from corner portions and are smoothed, an original contour of the image is lost in the case of 
conputer graphics and the image becomes out of focus in the case of photograph. 

[0100] On the other hand, in the cubic method, the interpolation function gradually approximates the lattice point 
between the adjacent points (t«0 and 1), generating an upward convexity. Further, the interpolation function becomes 
downwardly concave between adjacent points (t»1 and 2). In other vrords. one edge portion is varied so as to have such 
a difference as not to produce a step. As a result, the sharpness is increased and no step is produced in the photo- 
graph. However, since an edge does not have any analogue change in the computer graphics, the cubic method is 
unsuitable therefa. 

[0101] An interpolating process of pattern matching will now be described. 

[0102] Fig. 14 shows a character image written on the color information virtual cfrawing screen. The character is 
also conrposed of dot-matrix pxels arranged in the horizontal and vertical directions, and each dotted portion (•) is an 
image pixel, whereas each blank portion ( O) is a background pixel. 

[01 03] In the pattern matching, one area comprising sixteen pixels serving as a square area of 4x4 pixels as shown 
in Fig. 15 is matched with a previously prepared pattern data, so that interpolating pixels are generated with respect to 
a square area comprising inner four pixels (2x2 pixels). Outside pixels are also referred to in spite of the square area of 
four pixels. The reason for this is that the result of interpolation for the square area of four pixels changes depending on 
whether peripheral pixels exist. Fig. 15 shows two pattern data, that fe. the area matches the pattern data when the area 
comprises four pixels, whereas the area differs from the pattern when the area comprises sixteen pixels. In pattern data 
A. dots are arranged vertically with one dot projecting adeways. In pattern data B. three of four dots are dotted. In pat- 
tern data A, it is preferable that dots are ananged into an angled shape so that an image of projection is represented. 
When three pixels are added, they are preferably added so as to represent a triangle. Accordingly, an interpolation pixel 
pattern differs according to the pattern data. 

[01 04] A plurality of sets of interpolation pixel patterns are prepared for every scale factor. Fig. 1 6 shows an exam- 
ple in which the scale factor is 1.5 in the vertical and horizontal directions. 

[01 05] When the pattern matching is applied to color data, a large number of pattern data should be prepared even 
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in the example of four pixels. More specifically, the pattern matching results in combinations the number of which cor- 
L^*^ to pernujtations of the number of colors each pixel can take. In the embocfiment. however, pattern comparison 
s earned out on the basis of presence or absence of dot and the color data is coped with by color allocation, whereupon 
tne above-descn*bed problem is solved. Rg. 1 7 shows an example of such a case. The color data is compared with the 
5 pattern data compnsing sixteen pixels as in the above-described example. Concerning four pixels, a color of each pixel 
IS correlated with the interpolation pixels. Consequently, no previous processing for determining the colors of the inter- 
polafton pixels is required, and the number of pattem data is reduced, so that an amount of processing and an amount 
of resource are considerably reduced. 

[0106] Figs. 18A to 18C show a more concrete technique for pattern matching on the basis of sixteen pixels Rg 
10 16A shows an arrangement of original pixels to be interpolated. The pattern matching is executed with a small area of 
sixteen pixels is displaced. The pattern matching is can-ied out while a small area of the sixteen pixels is displaced. In 
this case, not all the information about the sixteen pixels need be renewed every time the small area is moved In Fig 
18A. object pixels are represented as "E. F, Q. H. I. J. K. L M. N. O. P. Q. R. S. and T" Rg. isB shows a data register 
area of the CPU etc. for processing the small area. Whether each pixel is dotted is represented by one-bit of "1" or "0 
15 so that the pattern matching can be carried out when data width of sixteen bits is provided. When the small area is 
moved by one pixel, four pixels of "A. B, C and D" are newly added to ttie small area, whereas four pixels of "Q, R, S 
and r are excluded from the small area, as showvn in Fig. 18C. Thus, the data register area is shifted by four bits and 
four bits conrespondcng to the four pixels of A, B. C and D' are introduced at LSB side. 

101 07] Further, when sixteen bits are utilized as an address concerning the arrangement of pattern data, the match- 
20 ing process includes only an addressing process, whereupon the interpolation pixel information can be obtained. 

[0108] On the premise that the aforesaid interpolating process is executable, the image data is divided into a type 
belonging to a natural image and a type belonging to a non-natural image. The processing as shown in Rg. 19 is earned 
out for the former, whereas the processing as shown in Rg. 20 is canrled out for the latter. Rgs. 21 A and 21 B show a 
case where image data is read out by one line for every type of image. The pixels are sorted out for every type of a nat- 
2S ural image, character and business graph while it is determined whether each pixel on the color information virttjal 
drawing screen is a natural image, character or business graph on the basis of attribute information virtual drawing 
screen. Pixel information previously initialized as background pixels is applied when sorted. 

[0109] Only the an^angement of pixels in ttie horizontal direction is insufficient for execution of the interpolating 
process, and information of pixels in the vertical direction is also required. Accordingly, four lines of pixels are actually 

30 read out to be stored in a work area, so that the interpolating process is earned out. Employment of four lines in ttie 
embodiment is based on the fact that sixteen pixels {4x4 pixels) are treated as a processing unit in the cubic method 
and the pattern matching. The number of lines may be increased or reduced as the need arises. 
[0110] A margin extending process is carried out for the natural irmge at step ST202. A margin is previously 
extended at the peripheral edge of a pixel in ttie margin extending process. Fig. 23 showvs ttie necessity of the process. 

35 [01 11 ] First, a region of natural image and a region of non-natural image are mixed so as to be adjacent to each 
other as shown in Rg. 23A. However, when ttie image data is divided for every type, a blank region is produced as 
shown in Fig. 23B. At tfiis time, the blank is processed merely as a background color. 

[0112] On ttie ottier hand, ttie cubic mettiod utilizes the cubic function so ttiat ttie interpolation pixel is smoottily 
changed. Accordingly, information about pixels outside a region to be interpolated is used in that region. Concerning 

40 the square region of sixteen pixels, when pixels are interpolated in ttie ireide square regfon of four pixels, information 
about outside twelve pixels is utilized. In short, outside information is brought in at ttie time of interpolation. 
[0113] This resutts in no problem inskle a natural image. However, the background pixels are generated In a mar- 
ginal portion adjacent to the background pixels. Accordingly, information about nothing or information about white or 
black is utilized in the interpolating computation. Fig. 23C shows ttiat information about background pixel generated in 

45 ttie margin of the region of natural image is brought into ttie adjacent pixel, so ttiat the interpolating computation influ- 
enced by ttie brought informatfon is carried out for the marginal pixels of interpolated ones. Since ttie aforesaid other 
method does not bring in the outside information, the influence upon ttie margin need not be considered as shown in 
Figs. 23D and 23E. 

[0114] Figs. 24A to 24D show a countermeasure in the case where ttie outside information is brought in. In Rg. 

so 24A. ttiree (A to C) of nine pixels contain pixel information and ttie ottier six pixels constitute background pixels. One 
Pixel adjacent to the margin is copied outside ttie margin such ttiat the margin is extended, as shown in Fig. 24B. Infor- 
mation about outside background pixels is not brought in when interpolation pixels are generated in original regions 
after ttie marginal extension (Fig. 24C). This example supposes the case of ttie cubic method, and ttie margin is 
extended outward by one pixel. However, the margin may be extended by tfie number of pixels necessary in ttie inter- 

55 polating process. 

[01 1 5] The process for extending ttie margin has two sides. One side is directed to not bringing in ttie information 
about background pixels adjacent to ttie margin in ttie interpolating process bringing in ttie pixel information from ttie 
outside region. The other side is directed to expanding ttie margin. Rg. 24D shows an example in which after extension 
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^lnrfr?2r"„rr, '"*®^'3«"9 Pracess is carried out for the pixels iriduding extended ones. When the margin is 
Sr^'^ S Jf generated in a portion which should originally be left as background pixels. This results in inva- 
nrr^al^^l ^ 1="^^® u^^'°" ^" ^^jacent Image regions changes. However, the 

Sn oven^riting the result of interpolating process carried out for the other image 

. ^ ^"^^ ^ extending the margin is to make a foundation. Referring to Figs. 23A to 23E. for example 
la^Ji norvnatural image are adjacent to each other when they are co-existent and accordirSyT™^ 
gaps are produced. However, the interpolating process may change a marginal contour. This change may result from a 
problem in the computation or an Influence of the pattern matching. Since the image data is smoothed according to an 
irterpola^ing sraile factor partcularly in the pattern matching as shown in Fig. 25, matching of an image region before 
interpolation with an adjacent image region is not taken into consideration. Accordingly, there is a case where the mar- 

^ *® ^^"^ "^"^^ correspond with the margin of an interpolation 

pixel obtain^ by another interpolabng process. In this case, a background pixel resulting in a gap is generated When 
a background pixel is generated where no background pixel is existent, only that pixel becomes white. In such a case 
me region one image is previously enlarged. Even in a case where a background pixel is generated in the marairi 
when the mterpolatng process has been carried out for the other image, the background pixel is prevented from 
appearing as a gap since pixels of the adjacent image region are generated in the foundatton 

.u ®*«'*""9 P^'^ess is carried out in the fbrmer sense at step ST202. and the interpolating proc- 

e^ by the cubic method is carried out at step ST204. Thus, the pixels of the natural image are distinguished and Vead 
out and the interpolating process suitable for the natural image can be carried out 

SiUjfl -ru °" '"«9e is divided into a character image and a business graph at step 

ST302 The interpolating process by the pattern matching is carried out for the character image at step ST306 and the 
interpolating process by the nearest method is canied out for the business graph at step ST304. The pattern retching 
is carried out for the character image so that an attractive margin contour is obtained. Regarding the business graph 
and computer graphics, since an original margin contour is preserved, a suitable interpolating process can be carried 
oul ln the business graph and computer graphics, there is a case where maintaining the original mai^in contour is pref- 
erable, whereas there is another case where a smoothed margin is preferred. Accordingly, the correspondence 
between the image data type and the interpolating process may be selectable. 

[0119] The natural image is firstly written at step ST206 after completion of the interpolating process, and the non- 
natural image is secondly written at step ST308 after completion of the interpolating process. The steps ST206 and 
ST308 are shown by chain line in the flowcharts of Rgs. 19 and 20 respectively. These processes actuaUy correspond 
to an interpolated image supwposing process alstepSTIOS in the flowchart of Fig. 4. 

10120] The pixels of the natural image and the pixels of the non-natural image are divided and interpolated in the 
disaete worii areas. When the natural and non-natural images are contained, the natural image is firstly written and the 
non-natural image is secondly written. In this case, the f irstly written pixels remain when the firstly and secondly written 
pix^are not superposed. The secondly written pixels remain when the firstly and secondly written pixels are siper- 

[0121] The above-described writing order is one mode of the superposing process. In this example, the interpolat- 
ing process by the pattern matching is carried out for the secondly written pixels. The pattern matching puts emphasis 
on smoothing the margin as shown in Fig. 25. Accordingly, a benefit of the pattem matching can effectively be obtained 
even when the marginal contour is not smoothed in the firstly written pixels. Further, the margin extending process is 
earned out in the firstly written pixels so that the margin is extended. The interpolating process is carried out for the sec- 
ondly wntten pixels without marginal extension. In this case, when the superposition of the pixels Is carried out, the 
foundation can effectively be prevented from being exposed. Thus, when the superposition at the maioinal portion is 
considered according to the nature of the interpolating process, the foundation can be prevented from being exposed 
or the marginal contour can be rendered fine. These processes constitute a margin processing unH and a superposing 
unit (synthestting unit) respectively. r- » k a 

[0122] A control manner for the above<lescribed writing order is varied on an actual program. Accordingly, only the 
wrrbng order needs to be maintained in any way and steps ST206 and ST308 are shown by chain line in this kense. 
[0123] When the interpolated pixels are superposed, a cotor correction is carried out for conversion of cotor coor- 
dinates from RGB to CMYK at step ST110. A half tone process is carried out at step ST112 since the color printer 17b 
has two gradations. Print data is output to the cotor printer 1 7b at step ST1 1 4. 

[0124] The fbregoing description relates to the printer driver 12c and can also be applied to the display driver 12b 
When a plurality of types of image data are superposed and j . 

color information virtual drawing screen, attribute information of each pixel is written on the 
attribute information virtual drawing screen so that the image data are distinguished for every type and read out When 
the result of superposition is read out from the virtual drawing screen and the interpolating process is carried out the 
image data is read out for every type thereof on the basis of the attribute information, so that the interpolating process 
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thri i^c«l^^ isramed out Further, since the margin of the image data is affected by the interpolating process. 
^LnSSf.!^l o 'nterpolating process is controlled so that a most suitable margin is obtained 

consequently, an opbmum result of interpolation can be achieved even when the image data are co-existent 
5 L^flor J« embodiment, the types of image data include the natural image and non-natural image. 

^ metacommand and non-metacommand and processing for them can be accwn- 

[01271 Fig 26 is a block diagram showing the image data interpolating apparatus of another embodiment in accord- 
ance with the invention. 

[0128] In this embodiment, in view of the difference in the natures of the images, the virtual drawing unit C21 inputs 
TO image data conresponding to metacommand and another image data and superposes the image data in a predeter- 
mined order so that both image data are distinguishable from each other, drawing on the virtual region. The readout unit 
C22 reads out the image data and a non-metacommand pixel interpolating unit C23 can-ies out an interpolating process 
so that the pixel corresponding to the image data other than the metacommand in the virtual region takes a predeter- 
mined interpolating scale factor. At this time, the peripheral region is extended and the Interpolating process is carried 
IS out. Accordngly. an interpolated image having a larger region than the original is obtained. On the other hand a meta- 
command pixel interpolating unit C24 also carries out an interpolating process so that the pixel corresponding to the 
image data other than the metacommand in the virtual region takes a predetermined interpolating scale factor. How- 
ever, the metacommand pixel interpolating unit C24 generates an interpolated pixel con-esponding to the original met- 
acommand. A superposing unit C25 synthesizes the result of interpolation by the non-metacommand interpolating unit 
so C23 and the result of interpolation by the metacommand interpolating unit C24. Regarding an overlapped portion the 
result of interpolation by the metacommand pixel interpolating unit C24 has priority to the result of interpolation by the 
non-metacommand pixel Interpolating unit C23. 

[0129] In the embodiment, the display driver 12b and the printer driver 12c constitute the virtual drawing unit C21 , 
readout unit C22. non-metacommand pixel interpolating unit C23. metacommand pixel interpolating unit C24 and 
25 superposing unit C25. 

[0130] Hg. 27 is a flowchart showing the resolution conversion canried out by the printer driver 1 2c. The image data 
IS input and sorted according to the superposition at step ST402. The rearranged image data is written onto the virtual 
drawing screen serving as the virtual region at step ST404. Fig. 28 typically shows writing onto the virtual drawing 
screen. The attribute infomiation virtual drawing screen is used so that each pixel is determined to be a non-metacom- 
mand pixel (N). or a character (C) or a business graph (B) of the metacommand pixel. One byte is allocated for every 
pixel so that an attribute identification code (N. C or B) is written. More specrtically. bit-map image data is processed as 
the non-metacommand pixel. The bit-map image data includes computer graphics as well as photograph. The bit-map 
image data may be divided into them and the interpolating process may be changed accorcling to them, or a predeter- 
mined interpolating process may be applied uniformly. Whether the image data is the photograph or computer graphics 
can be determined by analyzing the image data. As one example, the image data may be determined to be a natural 
image when the number of used colors is large, whereas the image data may be determined to be a non-natural image 
when the number of used colors is small. In the case of a natural image, even when objects have ttie same color, they 
are detemnined to have a plurality of colors due to shading. The relationship between the color information virtual draw- 
ing screen and the attribute information virtual drawing screen is the same as shown in Rgs. 6A and 68. 
[01 31 ] At step ST406. the image data is read out from the virtual drawing screen as shown in Rg. 28 for every type 
of image and an optimum Interpolating process according to tiie image type is carried out. 

[0132] As an interpolating process suitable for a natural image such as photograph, which Is an exanple of non- 
metacommand pixel, the interpolating process by a cubic method is executed. As an interpolating process suitable for 
a non-natural image such as computer graphics, which is another example of ttie non-metacommand pixels, ttie inter- 
polating process by a nearest metfiod is executed. On tfie other hand, the interpolating process by the pattem matching 
is can-ied out for tiie metacommand pixel. 

[0133] Pixels to be written are specified regarding the non-metaconnmand image when the drawing function 
according to tfie image data is called and data is written on the cotor information virtual drawing screen and the attribute 
information virtual drawing screen for every pixel. Regarding the metacommand pixel, the drawing function obtains pix- 
els to be written by computation. For the reason of these natures, parameters of the command are changed at the appli- 
cation 12d side so that a figure with any size can be drawn. However, when a plurality of commands con-espond to one 
figure as in a character, a pixel serving as noise is sometimes generated. More specifically, a dot is not produced at one 
scale factor while a useless dot is produced at another scale factor. The dot tends to be produced particulariy at a joint 
of lines drawn with a plurality of commands. It is preferable ttiat no such dot as noise is produced. 
[0134] Figs. 29A to 29D show a manner of eliminating noise dot employed in the embodiment. Suppose that an 
image shown in Fig. 29A is original. The image shows a part of a character and is found to be a corner of a frame- 
shaped figure. Further, the in^ge is supposed to be a joint of a plurality of drawing commands. However, tfie image is 
not necessarily a joint 
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Description 

FIELD OF THE INVENTION 

The present invention relates to an image process- 
ing method and an image processing apparatus which 
are used for an image input/^output device such as a dig- 
ital copying machine and reads an image and divides 
the read image into blocks composed of a plurality of 
picture elements so as to perform a variable scaling 
process on the image by interpolating or thinning out 
each picture element 

BACKGROUND OF THE INVENTION 

Conventionally, in the case where a variable scaling 
of an irput image Is performed by a digital copying 
machine, namely, the input image is enlarged or 
reduced by a digital copying machine, it is enough to 
select either of a nearest neighbor interpolation and 
y interpolations of first through third degrees per page. 
In the nearest neighbor interpolation, an image is 
divided into blocks cooposed of a plurality of picture 
elements, and when enlarging the image, the data of 
the picture element existing in the nearest neighbor- 
hood of a target picture element is used as data of the 
target picture element The enlarging method by means 
of the nearest neighbor interpolation has a preferable 
characteristic that an edge of a character becomes 
clear. 

In addition, in the interpolation of first through third 
degrees, data of a target picture element in divided 
blocks are determined by using an conputing mean of 
the data of the nearest neighbor picture element of the 
target picture element and data of another neighbor pic- 
ture element, which is obtained by formulas of the first 
through third orders so that an image Is enlarged. This 
method is also called as a linear interpolation. More 
specifically, the data of the nearest neighbor picture ele- 
ment of the target picture element and the data of 
another neighbor picture element are respectively 
weighted with an immerse number of a distance from the 
target picture element and a weighted mean is 
obtained. The data obtained from the weighted means 
is specified as data of the target picture element 

The enlarging method through the interpolations of 
first through third degrees is preferable as the enlarging 
process in the case of processing an image in which 
density gradually changes like a photographic image 
because new data of a picture element interpolated 
between the respective picture elements (interpolation 
picture element data) become mear^ of picture element 
data existing on both sides adjacent. 

However, jrf the case where two or more types of 
rmage data, such as characters, photographs and mesh 
dots, coexist on one page of image, if a variable magni- 
fication of such an image is processed by either of the 
nearest neighbor interpolation and the interpolations of 
first through third degrees, image quality of eHher of the 



characters, the photographs, etc. is deteriorated. 

Therefore. Japanese Unexamined Patent Publica- 
tion No: '6-286221/1994 (Tokukaihei 6-286221) dis- 
closes a printing method which is capable of changing a 
s vanable scaling process according to types of data in 
the case where characters, photographs, mesh dots 
etc. coexist on one page. 

In this printing method, input data such as character 
information such as character codes and form informa- 
10 tion are inputted from a host as an external device into 
a buffer memory of a printer. The printer creates a char- 
acter pattern and a form pattern according to the Infor- 
mation of the inputted data so as to print an Image. 
In the case where enlarging or reducing process is 
/ff executed prior to printing, when a special enlarging or 
reducing method is cataloged into the printer according 
to the character pattern, eta. variable scaling is per- 
formed by the enlarging or reducing process. Therefore, 
the variable scaling process can be performed accord- 
so ing to each type of data. 

However, since the above conventional image 
processing method is appli^ to a printer, and the varia- 
ble scaling process of such a printer can be previously 
judged by information from an external host that charac- 
25 ters. vector drawings and bit images is inputted as 
image data. Therefore, in image input and output 
devices such as a copying machine, the above-men- 
tioned image processing method has a disadvantage 
that in tfie case where document image is read by a 
30 scarcer and characters, photographs and mesh dots 
coexist on one page of the read document image, a var- 
iable scaling process cannot be performed according to 
the type of image data. 

35 SUMMARY OF THE INVENTION 



The present invention is invented from a viewpoint 
of the above conventional problem, and it is an object of 
tile present invention to provide an image processing 
40 nr>ethod and an image processing apparatus which are 
capable of, even if characters, photographs and mesh 
dots coexist in an image read by a scanner, preventing 
a deterioration in image quality by variable scaling the 
image according to the characters, photographs and 
45 mesh dots. 

In order to achieve the above object, an image 
processing method of the present invention for scaling 
an image is characterized by having: the step of dividing 
an image into blocks conrposed of a plurality of picture 
so elements so as to detect region segmentation data, 
which represent possibilities of characters, photographs 
and mesh dots in a block of a target picture element, by 
region segmentation means per target picture element; 
and the step of computing density of the target picture 
55 element on tine output image by using conputing means 
according to an equation into which density of a plurality 
of adjacent picture elements in tiie vidnity of the target 
picture element is inputted, wherein in the computing 
step, a weight of the density of each adjacent picture 
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element in the equation is adji^ed according to the 
region segmentation data of the target picture element. 

In other words, in the case where characters and 
photographs, etc. coexist in a document image, when 
the conventional nearest neighbor interpolation or the s 
interpolation of first through third degrees is used for 
enlarging magnification, the following problems arise. 
Namely, when the nearest neighbor Interpolation is 
selected, a pseudo-contour occurs on a photograph, 
and when the interpolation of first through third degrees io 
is selected, an edge of characters becomes unclear. 

Howeven in accordance with the above method, 
region segmentation data, which represent possibilities 
of characters, photographs and mesh dots of a target 
picture element on an image are detected by the region is 
segmentation means, and interpolated picture element 
data per picture element are computed by the conrput- 
tng means according to the equation in which density of 
a plurality of adjacent picture elements In the vicinity of 
the target picture element is inputted. At this time, a 20 
weight of the density of each adjacent picture element in 
the equation Is adjusted based upon the result detected 
by the region segmentation means. 

Therefore, a portion where the possitnl'ity of charac- 
ters is strong is enlarged by the nearest neighbor inter- 2s 
polation or the tike so that the edge of the enlarged 
characters can be prevented from becoming unclear. 
Meanwhile, a portion where the possibility of photo- 
graphs is strong is enlarged by the irrterpolation of first 
through third degrees or the like so that the pseudo-con- 30 
tour can t>e prevented from occurring on the enlarged 
photograph. 

As a result, even if characters, photographs and the 
mesh dots coexist on an image read by a scanner, the 
image is scaled according to the characters, photo- 3S 
graphs or mesh dots so that deterioration of image qual- 
ity can be prevented. 

It is desirable that the image processing method is 
such a method that in the detecting step, the regim seg- 
mentation data are set so as to take a value X which 40 
falls the range of 0 to N-1 (N Is an integral number not 
less than 2) and so that as the possibility of characters 
is stronger, the region segmentation data takes a 
smaller value and as the F>ossibilities of photographs 
and mesh dots are stronger, the region segmentation 45 
data takes a larger value, and in the computing step, 
density Dp of a picture element P which is a target pic- 
ture element whose density should be determined, is 
computed according to the following equation: 

so 

Dp«(1.K)xD^ + KxD(, (1) 

(However, K o (Xp/(N-1)) x (PA/(PA+PB)) ) 
where Xp is region segmentation data of the picture ele- 
ment P. Da is d^sity of a picture element A which is the ss 
closest to the picture element P. Db is density of a pic- 
ture element B which is the second closest to the picture 
element P. PA is a distance between the picture element 
P and the picture element A, and PB is a distance 



between the picture elemem P and the picture element 
B. 

In other words, characters, photographs and mesh 
dots mostly coexist on an actual image, and when pat- 
tern matching or the like is used as a variable scaling 
method for a target picture element of such an image, a 
lot of memories are required for performing the variable 
scaling process delicately, and time required for the 
process is increased. 

However, in accordance with the above method, the 
computing means computes the density Dp of the pic- 
ture element P according to the equation (1). In the 
equation (1), In the case where the region segmentation 
data Xp obtain a value 0 representing conplete charac- 
ters, for example, KsOandDpsD^. Namely, this 
means the variatrle scaling process by the nearest 
neighbor interpolation. 

Meanwhile, in the case where the region segmenta- 
tion data Xp obtains a value N-1 representing complete 
photograph, for example, 

Dp = (PB'Da+PA •Dt,)/(PA + PB) 

Therefore, Dp becomes a linearly weighted mean of the 
density D^ of the picture element A which is the closest 
to the picture element P and the density D5 of the pic- 
ture element B which is the second closest to the picture 
element P. 

Therefore, even if characters, photographs and 
mesh dots coexist on an image, when the nearest 
neighbor interpolation or the like is used for a portion 
where the possibility of characters is strong and a 
method which Is closer to the interpolation of first 
degree is used for a portion where the possibility of pho- 
tographs is strong at the time of perfonning the variable 
scaling pn>cess, a more suitable density of scaling- 
processed data can be obtained according by a single 
summing and multiplying operation of the equation (1) 
based upon the region segmentation data which repre- 
sent the possibilities of characters, photographs and 
mesh dots detected by the region segmentation means. 
Namely, since density can be selected by outstanding 
software suitably for the characters, photographs and 
mesh dots per target picture element, density can be 
determined quickly and the deterioration of image qual- 
ity can be prevented by a simple arrangement of hard- 
ware. 

In addition, it is desirable that the image processing 
method further has: the step of obtaining picture ele- 
ment density slope data which r^resent density slope 
of the picture elemerrt P with respect to picture elements 
which are adjacent to tiie picture element P after the 
conrputing step; and the step of on the picture element 
P whose region segmentation data Xp is 0 and whose 
density Dp is half-tone density, and on the picture ele- 
ment A which is the closest to the picture element P. 
when the picture element density slope data are data 
representing a relationship Da < Dp < D^. converting the 
density Dp into a value obtained by multiplying the half- 
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tone density and magnification of the variable scaling 
together, and setting the density Da to 0. whereas when 
the picture element density slope data are data repre- 
senting a relationship < Dp < D^. converting the den- 
sity Da into a value obtained by multiplying the half-tone 5 
density and magnification together, and setting the den- 



sity Dp toO 

In the case where tfie read image is character data 
and picture elements in the read position is an edge of 
the character, the density of tWs picture elernen is 10 
mostiy haff-tone density. Thea when the image is sub- 
ject to the variable scaling process by the summing and 
multiplying operation in the equation (1). the half-tone 
density continues, and thus the edge becomes unclear. 

However, in accordance witii the above method, 15 
when the density Da of the picture element A which is 
the closest to the picture element P is half-tone der^ity 
(i.e., larger than 0 and smaller than the maxirmim den- 
sity) and ttie picture element density slope data Sp of 
the picture element P are data representing a relation- so 
ship Da < Dp < D5 (positive), the density Dp is converted 
into a value obtained by multiplying the haff-tone density 
and magnification together and the density Da is set to 
0. whereas when the density Da is half-tone density and 
the picture element density slope data Sp of tiie picture 25 
element P are data representing a relationship D^ < Dp 
< Da (negative), the density Da is converted into a value 
obtained by multiplying the half-tone density and magni- 
fication together, and the density Dp is set to 0. 

Therefore, tiie density of two picture elements in the 30 
edge portion of the character data (pulse width for out- 
putting a laser) is subject to the variable scaling process 
so that the density of the picture element inside a. char- 
acter is varied according to magnification and the den- 
sity of the picture element outside the character 35 
becomes 0. As a result, the edge portion of the charac- 
ter data is further enhanced, and thus the edge portion 
of the characters can be prevented from becoming 
unclear. 

It is desirable that the image processing mettiod 40 
further has: the step of outputting a laser according to 
density data with respect to the target picture element 
after tfie converting step, wherein In the laser outputting 
step, a laser is outputted to the target picture element, 
to which the value obtained by multiplying the half-tone 4S 
density and the magnification of the various scaling 
together is given, in a position which is shifted to a side 
of thepicture element where the density is higher based 
upon the picture element density slope data of tiie tar- 
get picture element 

In otfier words, in the case where the read image is 
character data and the picture element in the read posi- 
tion IS an edge of the image, when the picture element 
which was subject to tiie process of. for example 8 
bits/kJicture element, has half-tone density, the picture ss 
element has the density of 128. i.e., 80H (H represents 
hexadecimal notation). Therefore, in an image output 
device which outputs a half-tone image by varying a 
pulse width of one picture element the inage data of 



80H are generated in the central position of the picture 
element for half dot of one picture element. Then in tfie 
case where the picture element, on which the image 
data tor half dot are generated in its central position is 
subject to the enlarging process, a white picture ele- 
ment of half dot appears between ttie picture element 
and an adjacent picture element Such a white picture 
element continues per picture element, and thus a so- 
called ghost contour occurs. 

However, in accordance witti the above metiiod a 
teser IS outputted for a target picture element, to whi'ch 
the value obtained by multiplying the half-tone density 
and the magnification together is given, in a position 
which is shifted to a side of the picture element where 
the density is higher based upon the picture element 
density slope data of the target picture element 

Therefore, when the picture element for the edge 
portion of the character data is subject to the variable 
magnification process according to magnification via 
tine image processing mettiod. tiie ghost contour is pre- 
vented from occurring on the left or right of one picture 
element at the edge of the character data. 

It is desirable that tiie image processing method is 
a such a method ttiat in tiie computing step, when the 
data of each interpolated picture element is conputed 
by the computing means, tiie region segmentation data 
detected by the region segmentation means is variably 
scaled so as to be outputted together witii the scaling- 
processed image data. 

In other words, in tiie case where the image data 
which was subject to variable scaling process are 
desired to be outputted into an extemal asynchronous 
image input/output device, such as a facsimile and a 
personal computer, together with the region segmenta- 
tion data, for example, if also the region segmentation 
data is not subject to the variable scaling process, a 
number of picture elements of tiie scaling-processed 
image data does not agree witti that of the region seg- 
mentation data. For tiiis reason, the region segmenta- 
tion data cannot be used. 

However, in accordance witti ttie above mettiod. the 
region segmentation data detected by the region seg- 
mentation means are also subject to ttie variable scal- 
ing process based upon tiie result detected by tiie 
region segmentation means so as to be outputted 
togetiier with tiie scaling-processed image data 

Therefore, a number of picture elements of ttie 
magnification varied image data can agree witti that of 
ttie region segmentation data. As a result, in tiie case 
also where ttiese data are outputted to an external 
asynchronous image input/output device such as a fac- 
simile and a personal computer, ttie region segmenta- 
tion data can be used. 

It is desirable ttiat ttie variable scaling process for 
tiie region segmentation data in the computing step is 
performed based upon ttie result detected by the region 
segmentation means. 

As a result, since the data representing ttie possibO- 
ities of characters, photographs and mesh dots are 
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given also to the scaitng-processed region segmenta- 
tion data, the scaling-processed innage can be output- 
ted to an external image input/output device such as a 
facsimile and a personal connputer by comtMning these 
data with the region segmentation cteta 5 

It is desirable that the image processing method 
further has: the step of simultaneously outputting the 
scaling-proc^ed image data and the scaling-proc- 
essed region siegmentation data after the computing 
step, wherein in the outputting step, directing data, 10 
which represent a front and a rear of each tine of plural 
picture elements on the output data and end of the 
image data, together with the scallng-processed image 
data and the ecaling-processed region segmentation 
data. 15 

In other words, in the case where the scaling-proc- 
essed image data are outputted to an external asyn- 
chronous image input/output device such as a facsimile 
and a personal computer, when the data, which repre- 
sent the front and rear of the line and the end of the so 
image data, do not exist in the magnification varied 
region segmentation data, a number of picture elements 
of one line and a total nuntber of lines should be trans- 
mitted to other external asynchronous image input/out- 
put device. Moreover, even in the case where this 25 
transmission is executed, a tine counter should be 
always provided. 

IHowever, in accordance with the above method, 
since the cfirecting data, which represent the front and 
the rear of each line of a plurality of picture element on 30 
the output data and the end of the image data, are 
simultaneousty outputted. a number of picture elements 
for one line and a total number of lines does not have to 
be transmitted to an external synchronous image 
input/output device. Furthermore, installation of a line 35 
counter can be avoided. 

An image processing apparatus of the present 
invention for scaling inputted image so as to output the 
scaling-processed image is characterized by having: 
region segmentation means for dividing the input image 40 
into blocks composed of a plurality of picture elements 
and detecting region segmentation data, which repre- 
sent possibilities of characters, photographs and mesh 
dots of a block of a target picture element per target pic- 
ture element; and computing means for computing derv 4S 
sity of the target picture element on the output image by 
using an equation in which density of a plurality of adja- 
cent picture elements in the vidnity of the target picture 
element is inputted, 

wherein the computing means adjusts a weight so 
of the density of each adjacent picture element in the 
equation according to the region segmentation data of 
the target picture element. 

In accordance with the above arrangement, the 
region segmentation data, which represent the possibih ss 
ities of characters, photographs and mesh dots on the 
image, are detected by the region segmentation means, 
and the irtterpolated fMcture element data per picture 
element are connputed by the conputing means accord- 



ing to the equation in which the density of a plurality of 
acflacent picture elements in the vicinity of the target pic- 
ture element is inputted. At this time, a weight of the 
density of each adjacent picture element in the equation 
can be adjusted based upon the result detected by the 
region segmentation means. 

Therefore, the portion where the possibility of char- 
acters is Strong is enlarged by the nearest neighbor 
interpolation or the like so that the edge of the enlarged 
c^iaracters can be prevented from becoming unclear. 
Meanwhile, the portion where the possibility of photo- 
graph is strong is enlarged by the interpolation of first 
through third degrees or the like so that a pseudo-con- 
tour is prevented from occurring on the enlarged photo- 
^aph. 

Asa result, even in the case where characters, pho- 
tographs and mesh dots coexist in an image read by a 
scanner, deterioration of image quality can be pre- 
vented by the variable scaling process according to the 
characters, photographs and mesh dots. 

For fuller understanding of the nature and advan- 
tages of the invention, reference should be made to the 
ensuing detailed description taken in conjunction with 
the accompanying drawings. 

BRIEF DESCRIPTION OF THE DRAWINGS 

FIG. 1 is a block diagram which shows an arrange- 
ment of a digital copying machine as an inrege process- 
ing €q3paratus which adopts an image processing 
method according to one embodiment of the present 
invention. 

FIG. 2 is an explanatory drawing which explains an 
image processing function of the digital copying 
niachine. 

FIG. 3 is an explanatory drawing which shows a 
computing process of a variabje scaling section in the 
digital copying machine. 

FIG. 4 is a block diagram which shows an output 
control section and an image output section in the digital 
copying machine. 

FIGS. 5(a) through 5(c) are explanatory drawings 
which show output examples of a pulse width modulator 
on each picture element: FIG. 5(a) is a drawing which 
shows an output of the pulse width modulator in the 
case where half-tone data are outputted shifted to the 
left; FIG. 5(b) is a drawing which shows an output of the 
pulse width modulator in the case where half-tone data 
are outputted shifted to the right; and FIG. 5(c) is a 
drawing which shows an output of the pulse width mod- 
ulator in the case where half-tone data are outputted 
being centered. 

FIG. 6 is a graph which shows an original image 
having tone. 

FIG. 7 is a graph which shows density of read pic- 
ture elements when the original image shown in FIG. 6 
is read. 

FIG. 8 is a graph which shows an output in the case 
of using a nearest neighbor interpolation when the read 
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picture elements shown in FIQ 7 are enlarged by twice. 

FIG. 9 is a graph which shows an output in the case 
of using interpolation of first degree when the read pic- 
ture elements shown in FIG. 7 are enlarged by twice. 

FIG. 10 is a graph which shows read density data 
when an image in which characters and photographs 
coexist is read. 

FIG. 11 is a graph which shows an output in the 
case of using the nearest neighbor interpolation when 
the read picture elements shown in FIQ 10 are 
enlarged by twice. 

FIG. 12 is a graph which shows an output in the 
case of using the interpolation of first degree when the 
read picture elements shown In FIQ. 10 are enlarged by 
twice. 

FIQ. 13 is a graph which shows an output in the 
case of using the nearest neighbor interpolation in a 
section judged to be characters and using the interpola- 
tion of first degree in a section judged to be a photo- 
graph when the read picture elements shown in FIG. 10 
are enlarged by twice. 

FIG. 14 is a graph which shows read density data 
when an image in which characters and photographs 
coexist is read. 

FIG. 15 is a graph which showvs an output in the 
case of using the nearest neighbor interpolation when 
the read picture elements shown in FIG. 14 are reduced 
to half. 

FIG. 16 is a graph which shows an output in the 
case of using the interpolation of first degree when the 
read picture elements shown in FIQ. 14 are reduced to 
half. 

FIG. 17 is a graph which shows an output in the 
case of using the nearest neighbor interpolation in a 
section judged to be characters and using the interpola- 
tion of first degree in a section judged to be a photo- 
graph when the read picture elements shown in FIG. 14 
are reduced to 1/2. 

FIG. 18 is a graph which shows an output in the 
case of giving an edging process to a section judged to 
be characters in FIG. 13. 

FIGS. 19(a) through 19(c) are explanatory drawings 
which show effects of an ON signal position operation in 
the picture elements by means of the pulse width mod- 
ulator: FIG. 19(a) is a drawing which shows denaty val- 
ues (256 tones) of the picture elements; FIQ. 19(b) is a 
drawing which shows an output when a picture element 
density slope data is fixed to the center; and FIG. 19(c) 
is a drawing which showvs an output when the pulse 
width is outputted shifted to the side where the density 
is high based upon data representing the density slope. 

FIG. 20 is a block diagram which shows an arrange- 
ment of the digital copying machine which adopts an 
image processing method according to another embod- 
iment of the present invention. 

FIG. 21 is a block diagram which shows an an-ange- 
ment of a variable scaling section in the digital copying 
machine. 

FIGS. 22(a) and 22(b) are explanatory drawings 



which show states that when scaling-processed image 
data and scafing-processed region segmentation data 
are outputted simultaneously, their respective directing 
data are outputted simultaneously: Fia 22(a) is a draw- 
5 ing which shows the front of each line of each data 
inputted to the variable magnification processing sec- 
tion; and FIG. 22(b) is a drawing which shows the from 
Of each line of each data outputted from the variable 
magnification processing section. 
10 FIGS. 23(a) and 23(b) are explanatory drawings 
which show states that when scaling-processed image 
data and scaling-processed region segmentation data 
are outputted simultaneously, their re^ective directing 
data are outputted simultaneously: FIQ. 23(a) is a draw- 
15 ing which shows the rear of each line of each data input- 
ted to the variable scafing section; and FIG. 23(b) is a 
drawing which shows the rear of each line of each data 
outputted from the variable scaling section. 

FIGS. 24(a) and 24(b) are explanatory drawings 
20 Which show states that when scaUng-processed data 
and scaling-processed region segmentation data are 
outputted simultaneously, their respective directing data 
are outputted simultaneously: FIG. 24(a) is a drawing 
which shows an end portion of an image data of each 
25 data inputted into the variable scaling section; and FIQ. 
24(b) is a drawing which shows an end portion of image 
data of each data outputted from the variable magnifica- 
tion processing section. 

so DETAILED DESCRIPTION OF THE PREFERRED 
EMBODIMENTS 

[EMBODIMENT 1] 



35 The following describes one embodiment of the 
present invention on reference to FIGS. 1 through 17. 

As shewn in FIG. i , a digital copying machine as an 
image processing apparatus of the present embodiment 
is provided with an image input section 1 , a shading cor- 
^ recting/automatic exposing section 2, a region segmen- 
tation section 3 as region segmentation means, a 
variable scaling section 4 as computing means, a y cor- 
recting section 5, an output control section 6 and an 
image output section 7. 
45 The image input section 1 reads a document image 
from a scanner, not shown, so as to convert the read 
image into digital input image data la. The shading cor- 
recting/automatic exposing section 2 processes shad- 
ing correction and automatic exposure of the input 
so image data la. 

While referring to density, etc. of picture elements in 
the vicinity of target picture elements of the image data 
2a which were subject to the shading correction and the 
automatic exposure, the region segmentation section 3 
ss detects a possibility of characters, a possibility of photo- 
graphs and a possibility of mesh dots of the target pic- 
ture elements of the image data 2a so as to output 
image data 3a and region segmentation data 3b. 
The region segmentation data 3b, which represents 
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the possibilities of characters, photograph and mesh 
dots, take a value X which falls wrthin the range of 0 to 
N-1 (N: an integral nun^er not less than 2). The smaller 
the value X is, the stronger the possibility of characters 
Is. and the larger the value X is, the stronger the possi- 
bilities of photographs and mesh dots are. In the 
present embodiment, NsS. tor example, is adopted. 
Therefore, when the possibilities of photographs and 
mesh dots are the strongest, the region segmentation 
data 3b take the value Xo7. 

The following describes a technique of detecting 
the possibilities of characters, photographs and mesh 
dots in each picture element The possibilities of char- 
acters, etc. can be detected by the prior art, but in the 
preserrt embodiment a few improvements are made on 
the prior art 

As the prior art, for exanple, there exists an art of 
dividing an image into blocks composed of a plurality of 
picture elements and identifying the image per block by 
using pattern matching or feature parameters which 
r^esent properties of a character image and a mesh 
dot image so as to detect the possibilities of characters, 
etc. 

Since the image identification method using the 
pattern matching requires preparation of many patterns, 
there arises a problem that a memory capacity 
becomes enornrx}us and that this method lacks versatil- 
ity. For this reason, the image identification method 
using the feature parameters is adopted increasingly at 
present 

As the image identification method using the fea- 
ture parameters. Japanese Unexamined Patent Publi- 
cation No. 61-194968/1986 (Tokukaisho 61-194968) 
discloses a mesh dot photographic region identification 
method of individually measuring changes in signal lev- 
els of two picture elements which spatially continue in 
the two cases where the two picture elements continue 
In a horizontal scanning direction and continues in a 
vertical scanning direction and comparing sum totals of 
measured amounts In the blocks respectively with pre- 
determined values so as to identify the image according 
to the conr^ared results. 

In addition, as another method using the feature 
parameters, Japanese Unexamined Patent Pukslication 
No. 62-147860/1987 (Tokukaisho 62-147860) discloses 
a half-tone facsimile signal processing method. In this 
method, a difference between a maximum signal level 
and a mininrtum signal level in the blocks is obtained, 
and the difference value is compared with a predeter- 
mined value. Then, when the level difference is smaller 
than the predetermined value, a judging signal, which 
represents that the signal level in a section Including a 
photograph is changed mildly, is outputted. whereas 
when the level difference is larger than the predeter- 
mined value, a judging signal, which represents that the 
signal level in a section including contour of characters 
and a photograph or a mesh dot photograph is changed 
intensely, is outputted. Moreover, a number of changes 
between the two signal levels of the two picture ele- 



ments which continues spatially is compared with a pre- 
determined value according to order of access which is 
predetermined in the blocks, and according to the com- 
pared results, when the number of changes is larger 

5 than the predetermined value, a judging signal, which 
represents that, the block is a mesh dot section, is out- 
putted, whereas when the number of changes is smaller 
than the predeternined value, a judging signal, which 
represents that the block is not a mesh dot section, is 

10 outputted. Then, the picture elements in the blocks are 
subject to a signal process according to the respective 
judging signals. 

An image process which inrproves Image quality, a 
space filtering process Is conventionally used. As an 

IS image processing mettiod for inproving the image qual- 
ity by using the f Btering process, for exanple. Japanese 
Examined Patent Puhrfication No. 5-147860/1993 (Toku- 
kohei 5-147860) discloses a haH-tone facsimile signal 
processing method. In this method, a space filter which 

so makes an image signal smooth and a space filter which 
enhances an image signal are prepared, and a signal, 
which snnoothed the image signal and/or a signal which 
enhanced the image signal are/is mixed or selected 
based upon an output of edge detecting means which 

25 detects an edge section of the image signal. Moreover, 
another method is disclosed in Japanese Unexamined 
Patent Publication No. 63-246076/1988 (Tokukaisho 63- 
246076). In this method, a space filter, such as a filter 
processing device, which removes a mesh dot compo- 

30 nent is prepared, and when an edge section is not 
extracted by edge extracting means which extracts an 
edge section of an image signal, a signal, which has 
been subject to the filtering process for removing the 
mesh dot conrponent is outputted. whereas when an 

35 edge section is extracted, a signal, which has not been 
subject to the filtering process yet is outputted. 

However, tiie above-mentioned conventional image 
identification method has a disadvantage that misjudg- 
ment of the image identification occurs. 

40 The cause for the misjudgment of tiie image identi- 
fication is consklered to be unsuitable feature parame- 
ters, namely, the feature parameters do not sufficiently 
represent respective properties of the regions. Besides, 
unsuitable classifying method for the image identifica- 

45 tion according to an amount of features obtained by the 
feature parameters and unsuitable selection of a thresh- 
old value tor the classification also cause tine misjudg- 
ment of the image identification. 

In the conventional method of classifying and iden- 

50 tification the blocks according to an amount of features 
obtained by the feature parameters so as to execute the 
f Stering process on target picture elements in the blocks 
using a prepared space filter according to the identified 
results, great influence is exerted on image quality at 

55 the time of misjudgment. and an annount of features 
owned by the target picture elements reflects only lim- 
ited fitter characteristics. For this reason, there arises a 
problem that a delicate process such as the filtering 
process cannot be performed most suitably for the tar- 
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get picture elements. 

Therefore, in the present embodiment as disclosed 
in the U.S. patent application Serial No. 08/540,580 
(Japanese Patent Afplicalion No. 6-264232/1994) by 
the inventors of the present invention, an image signal 
obtained by scanning a document is subject to an iden- 
trfication process which identrfies as to whether each 
picture element exists in a character region, a photo- 
graphic region or a mesh dot region, and when each 
picture elemertt is subject to an Image process accord- 
ing to the results of the identification process, certain 
picture elemerrt data of the image signal is used as a 
target picture element, and image data in a local blodi, 
which composed of the target picture element and a plu- 
rality of picture elements in the vicinity of the target pic- 
ture element, are stored in a block memory. 

Next, according to the image data in the local block 
stored in the block memory, a plurality of feature param- 
eters, which respectively repr^ent features of the char- 
acter region, the photographic region and the mesh dot 
region, is obtained. A multi-dimensional identification 
process is executed by identification means which 
selects each of the character region, the photographic 
region and the mesh dot region divided by borderlines 
including a non-linear characteristics in a multi-dimen- 
sional plane including axis of the above feature param- 
eters and which is composed of a neural network which 
learned beforehand for receiving each feature of the 
plural feature parameters and outputting region Identifi- 
cation information according to each input. Namely, the 
identification process is performed not by setting a 
threshold value for each feature parameter but based on 
the borderlines including non-linear characteristic 
based on which the multi-dimensional space is divided 
by feature parameters. 

According to the described method, even if an iden- 
tification cannot be performed with a desired accuracy 
when considering only one feature parameter, by taking 
plural feature parameters Into consideration, the identi- 
fication of the region where the target picture element is 
located can be performed with an Improved accuracy. 
Furthermore, as a multi-dimenstonal identification proc- 
ess is performed using a neural network which receives 
each input of plural parameters, an identification can be 
performed with very high precision. 

In addition, the region identification information out- 
putted by the kJentif ication means is data wfiich respec- 
tively, represent likelihoods of the character region, the 
photographic region and the mesh dot region in a region 
of the block in which the target picture element exists by 
numerals. Namely, the identification means outputs a 
possibility of characters which is a numeral representing 
a character region in a region of a block where a target 
picture element exists, a posstoility of photographs 
which is a numeral representing a photographic region i 
in a region of a block where a target picture element 
exists, and a possibility of mesh dots which is a numeral 
representing a photographic region in a region of a 
block where a target picture element exists. 



According to the above manner, the possibilities of 
characters, photographs and the mesh dots in each pic- 
ture element are detected. Then. data, which are used 
for obtaining the possibilities of characters, photographs 
5 and mesh dots from the feature parameters (for exam- 
ple, a difference between the maximum density value 
and the minimum density value In a certain block tor pic- 
ture elements in the vicinity of the target picture ele- 
ment), are region segmentation data, and the region 
10 segmentation data are determined by the aforemen- 
tioned method. 

Then, as m^ioned above, in the present embodi- 
ment the above-mentioned data, namely, the region 
segmentation data 3b. can take a value X which falls 
IS within the range of 0 to N-1 (N; an integral number of not 
less than 2). 

In the space filtering process suggested in the U.S. 
Patent Application Serial No. 08/540.580. various filters 
in which filter coefficiems were predetermined are 
20 selected based upon the identification signal. More con- 
aetely, the filter coefficients are determined per proc- 
essed picture element based upon the data 
representing fikelihood of each region by numerals so 
that the space filtering process is performed. 
ss Next, the variable scaling section 4 performs the 
variable scaling process on the ir^utled image data 3a 
and region segmentation data 3b, namely, enlarges or 
reduces of the inputted image data 3a and region seg- 
mentation data 3b so as to output scaling-processed 
30 image data 4a which are scaling-processed image data, 
and magnification varied region segmentation data 4b 
which are scaling-processed region segmentation data, 
and judges slope of density of a current picture element 
by refering to peripheral picture elements so as to out- 
35 put Special data 4c. 

At the time of the variable scaling process, interpo- 
lation represented by an operation expression, men- 
tioned later, is used as the image data 3a, and nearest 
neigftoor interpolation is used for processing the region 
40 segmentation data 3b. Moreover, the special data 4c 
are composed of directing data, which represent the 
front and rear of a line of the region segmentation data 
3b and end of the image data, and data, which repre- 
sents slope of density of a picture element being proc- 
45 essed currently (i.e.. picture element density slope 
data"). 

In addition, the picture element density slope data 
are outputted as data representing "No slope** except 
tiiat the region segmentation data 3b has a strong pos- 
50 sibillty of characters. Namely, the variable scaling sec- 
tion 4 outputs *'No slope" as tiie picture element density 
slope data except that the region segmentation data 3b 
has a value representing a stot)ng possibility of charac- 
ters (here. 0). 

i5 Since the variable scaling process for an image in 
the vertical scanning direction of a digital copying 
machine is adjusted by changing a speed of an optical 
system, the image data 3a is subject to the variable 
scaling process by the variable magnification process- 
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ing section 4 in only the horizontal scanning direction of 
the digital copying machine. 

The y conceding section 5 performs the y correcting 
process on the inputted scaling-processed image data 
4a. 5 

The output control section 6 is also called as a 
pulse width modulator, and controls an output of an 
Image. The image output section 7 outputs a laser 
based upon a signal of the output image data 6a input- 
ted to the image output section 7. w 

In the digital copying machine of the pr^ent 
emt>odiment. the input image data 1a. the image data 
2a and 3a. the scaling-processed image data 4a and 
the scaling-processed y corrected image data 5a are 
respectively composed of 8 bits per picture element is 
(namely, each picture element takes a value of 256). 
Moreover, the output image data 6a is aggregate of sig- 
nals with 256 pulses per picture element. The output 
control section 6 aeates data as to how many pulses of 
256 pulses obtained by dividing one picture element 20 
into 256 is turned on. based upon a value of the inputted 
scaling-processed y corected image data 5a, and rep- 
resents density by outputting the created data. 

In addition, the region segmentaton data 3b and 
the scaling-processed region segmentation data 4b are 2s 
respectively corrposed of 3 bits per picture element 
(namely, each picture element takes a value of 8). Fur- 
ther, the special data 4c are conrposed of a total of 5 
bits: 2 bits of the directing data representing the front 
and rear of a line and end of image data; and 3 bits of 30 
the picture element density slope data in a picture ele- 
ment being processed cunrently. Moreover, the special 
data 4c are conribined with the scaling-processed region 
segmentation data 4b so as to take a value of 8 bits per 
picture element. 55 

The following describes an image processing 
method relating to the variable scaling process in the 
digital copying machine having the above arrangement 

As shown in FIG. 2, when density in a position P of 
a picture element after the variable scaling process. 40 
namely, output picture element density Dp is deter- 
mined, a simple summing and nrultiplying operation is 
performed by using a picture element A which has derv 
sity and region segmentation data X^. and is the 
nearest to the picture element P (i.e., "nearest neighbor 4S 
picture element^, a picture element B which has den- 
sity Db arKi region segmentation data Xb and is the sec- 
ond nearest to the picture element P (i.e., "second 
nearest neighbor picture element^, and distances PA 
and PB between the picture elements P. A and 6. so 

The at}ove summing and multiplying operation is 
performed in the variable scaling section 4. 

As shown in FIG. 3. the region segmentation data 
Xa and the density Da in the nearest neighbor picture 
element A. and the region segmentation data X^ and ss 
the density D^ in the second nearest neighbor picture 
element B are inputted into the variable scaling section 
4. Outputs of the variable scaling section 4 are output 
picture element region segmentation data Xp as the 



magnification varied region segmentation data 4b. out- 
put picture element density Dp. data representing a 
position of the output picture element used in the output 
control section 6. i.e.. the picture element density slope 
data Sp. and bits for controlling the directing data repre- 
senting the front and rear of a line and the end of an 
image data. 

In the variable scaling section 4. v^en the output 
picture element region segmentation data Xp is com- 
puted, a relationship Xp « X^ is fulfilled by the nearest 
neighbor interpolation. Namely, the nearest neighbor 
interpolation directly uses data of a picture element 
exist in the nearest neighbor as the output data of pic- 
ture element. 

Meanwhile, the output picture element density Dp is 
computed accorcfing to the following equation. 

Dp-(1.K)xD^ + KxDb (1) 

(Here. K - (Xp/N-1) x (PA / (PA + PB)) (2) 

In the present embodiment, since a number of sep- 
arations of the region segmentation data is set to 8, the 
value N-1 in the equation (2) is conrputed according to 
the equation N-1 e 8 - 1 =7. 

When the output picture element region segmenta- 
tion data Xp takes a value representing that the possibil- 
ity of characters is strongest, namely. Xp » 0. the 
equation (1) becomes as follows: 

Dp = D^ 

and thus the equation (1) represents the nearest neigh- 
bor interpolation. Namely, the nearest neighbor interpo- 
lation directly uses the den^ty of the picture element A 
as the density of the output picture element D. 

In addition, when the output picture element region 
segmentation data Xp takes a value representing the 
strongest possibaity of photographs, namely, 
Xp es N-1 s 7 . the equation (1) becomes as follows: 

Dp«(PB*Da + PA •Dtj)/(PA + PB) 

and thus the equation (1 ) represents the interpolation of 
first degree. Namely, the interpolation of first degree 
uses linearly weighted mean of weighing the density of 
the picture element A with the density of the picture ele- 
ment B as the density of the output picture element D is 
determined. 

Next, when Xp « 0. namely, the block of a picture 
element is judged to be characters in the region seg- 
mentation process, the relationship D^^ < Dp < D^^ is ful- 
filled. Moreover, when Dp is on the left side of D^, data 
representing the position of an output picture element in 
one picture element. i.e.. the picture element density 
slope data Sp the picture element density slope data Sp 
becomes data representing that a picture element is 
outputted in a position which is shifted to the left, 
namely, Sp & 01 B (B is binary display). 
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Meanwhile, when Xp « 0. > Dp > Db, and D is on 
the nght side of Dg. Sp becomes data representing that 
a picture element is outputted in a position which is 
shifted to the right, namely, Sp = 10B. Furthermore, 
when Xp = 0 and the condition of Da. Db and Dp is other 5 
than the above. Sp becomes data representing that a 
picture element is outputted in a central position, 
namely, Sp s COB. 

In addition, when Xp 0. namely, the picture ele- 
ment density slope data Sp are judged not to be charac- 10 
ters in the region segmentation process, Sp becomes 
data representing that a picture element is outputted in 
the central position, namely. Sp = COB . 

When output picture element P is a front or rear of 
a line or an end of a page, corresponding directing data 15 
are added to the picture element density slope data Sp 
so that the special data 4c are outputted. 

As shown in FIG. 1, the data of the output picture 
elements P. namely, the scaling-processed image data 
4a. the scaling-processed region segmentation data 4b so 
and the special data 4c are respectively inputted to the 
output control section 6 after the scale-processed 
image data 4a is subject to the y con-edion. 

The following describes a detailed operation of the 
output control section 6. 25 

As shown in FIG. 4. in the output control section 6. 
a pulse vifldth is modulated by the pulse width modulator 
16. 

Inputs of the pulse width modulator 16 are image 
data of 8 bits per picture element which are composed 30 
of the scaling-processed y corrected image data 5a and 
the magnification varied region segmentation data 4b, 
and picture element density slope data of 3 bits per pic- 
ture element which are included in the special data 4c. 

The image data and the picture element density 35 
slope data are synchronously inputted to the pulse 
width modulator in parallel. Outputs of the pulse width 
nroduiator 16 are serial video data which are divided 
into 256 per picture element The pulse width modulator 
1 6 control a laser of the image output section 7 accord- 40 
ing to pulses of an ON signal of 1/256 picture element 
based upon the input image data of 8 bits per picture 
element, namely, with value of 256. and controls the 
density of one picture element by 256 steps. 

In addition, the pulse width modulator 16 refers to 45 
the density slope data so as to determine a position of 
one picture element where continuous ON signals are 
allowed to be generated. Namely, when the picture ele- 
ment density slope data Sp of 3 bite per picture element 
are 018 which represents that they are shifted to the so 
left, the continuous ON signals are allowed to be gener- 
ated at the front of the picture element, and when the 
picture element density slope data are 10B which repre- 
sents that they are shifted to the right the continuous 
ON signals are aflowed to be generated at the rear of ss 
the picture element. Moreover, when the picture ele- 
ment density, slope data Sp are COB which represents 
that they are centered, the continuous ON signals are 
allowed to be generated at the center of the picture ele- 



ment 



More concretely, as shown in FIG. 5(a), when input 
image data are 80 H. i.e., have 128/256 density, and the 
input picture element density slope data of the special 
data are 108. namely, they are shifted to the left, half 
(128/256) picture element from the front of the picture 
element is turned on, and the rest of half picture ele- 
ment is turned off. 

In addition, as shown in FIG. 5(b). when input 
image data are 80 H, i.e., have 128/256 density, and tiie 
input picture element density slope data of the special 
data are 018, namely, they are shifted to the right, half 
(128/256) picture element from the front of the picture is 
turned off, and the rest of a half picture element is 
turned on. 

Furthermore, as shown in FIG. 5(c). input image 
data are 80 H. i.e., have 128/256 density, and the input 
image density slope data of the special data are 008. 
namely, they are centered, one-fourth (64/256) picture 
element from the front of the picture element is turned 
off, a next half picture element is turned on. and the rest 
of one-fourth picture element is turned off. 

In such a manner, the pulse width modulator 16 
adjusts outputs of the laser so as to change a position of 
ON data within one picture element 

The following details a variable scaling process of 
an image with variable density, i.e.. an image with den- 
sity tone according to the equation (1 ). 

An example is given as to an original image having 
density tone showvn in FIG. 6. A vertical axis represents 
the density of the original image. In ttiis case, when the 
original image is a white image, the density value is 0, 
and when tfie original image is a black image, the den- 
sity value is 255. Moreover, a horizontal axis represents 
a position where the above density is ol>tained. 

When an original image having density tone shown 
in FIG. 6 is read by the image input section 1 . as shown 
in FIG. 7, the ori^nal image is divided according to each 
picture element A vertical axis in FK3. 7 represente 
density values, which are integral numbers of 8 bits, i.e., 
0 to 255. On the horizontal axis, the position in FIG. 6 is 
divided by a unit of one read picture element, and rec- 
tangles of the graph r^ectively represent tiie density 
values of the read picture elements per read picture ele- 
ment unit. 

Next, when the read picture elements shown in 
FIG. 7 are enlarged twice by using the nearest neighbor 
interpolation, the read picture elemente are represented 
by FIG. 8. In FIG. 8, a vertical axis represents density 
values which are integral numbers of 8 bite, i.e., 0 to 
255. Moreover, a horizontal axis represente a position in 
a write picture element unit Further, rectangles of the 
graph respectively represent density values of a write 
picture element in a unit of one write picture element. 

Since the variable scaling process in this case is 
the nearest neighbor interpolation, data, in which a rec- 
tangle with the same shape is an-anged next to each 
rectangle of the read picture element unit shown in FIG. 
7. namely, data, in which two picture elemente having 
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the same density make a pair, are otrtained. According 
to this method, an image having a comparatively clear 
edge is obtained for a binary image such as characters. 
However, a variation of the density cannot be precisely 
reproduced tor an image such as a photographic image 5 
whose density is successively changed. As a result, a 
so-called pseudo-contour is caused. 

Here, the pseudo-contour is a contour which is 
caused when continuity of variation of density in a vari- 
able density image Is tost, and does not exist in an orig- 10 
inal image (see p. 461 "Handbook of Image Analysis", 
first edition published by the Publishing Foundattbn. the 
University of Tokyo: the supervisbn of Mikio Takagi and 
Shimoda). 

Therefore, as for an image such as a photographic is 
image whose density is successively changed, it is pref- 
erable that the variable scaling process is performed t)y 
using the interpolation of first degree. 

In other words, the original image read data shown 
in FIG. 7 are enlarged twice by using the interpolation of 20 
first degree, they are represented FIG. 9. 

According to the interpolation of first degree, a pic- 
ture element having density, which is the same as a 
mean of densities of two read image data, is interpo- 
lated between the two continuous read image data. 2S 
According to this method, as fbr an image such as a 
photographic image whose density is successively 
changed, the variation of density of the original image 
can be reproduced comparatively definitely and facth- 
fulty, t>ut as for a binary image such as characters, an so 
image whose contour is rxjt dear is reproduced. 

Therefore, in the case where characters, photo- 
graph, etc. coex^ in an image, if the nearest neighbor 
interpolation or the interpolation of first degree is 
selected indiscriminately, there arises a problem 35 

This is described concretely on reference to the 
drawings. 

Rrst, read density data which is obtained by read- 
ing an image In which ctraracters and photographs 
coexist are shown in FIG. 10, for exampla The left half 40 
of FIG. 10 Illustrates a character region, and Its feature 
is such that the density changes rapidly. On the con- 
trary, the right half of FIQ. 10 illustrates a photographic 
region, and its feature is such that the density changes 
continuously and comparatively mildly. 4S 

When the read image is enlarged twice by the near- 
est neighbor Interpolation, the tnnage represented by 
FIG. 1 1 is obtained. 

In this case, since the nearest neighbor interpola- 
tion is used, the edge of the left half, namely the char- so 
acter region of the image is comparatively emphasized, 
and thus the image becomes dear However, the right 
half, namely, the photographic region has unsatisfactory 
tone production, and thus a pseudo-contour is liable to 
be caused. 55 

Meanwhile, when the read image shown in FIG. 10 
is enlarged twice by the interpolation of first degree, the 
image represented by FIG. 12 is obtained. 

In this case, the interpolation of first degree causes 



an image whose left half, namely, character region has 
an unclear edge. However, the right half, namely, the 
photographic region has satisfactory tone production, 
and thus a pseudo-contour is difficuMy caused. 

In such a manner, it is not preferable that the near- 
est neighbor interpolation or the interpolation of first 
degree Is indiscriminately selected for a whde image. 

Therefore, in the present emt)odiment, in the case 
where characters and photographs coexist in a read 
image, the nearest neighbor interpolation or the like is 
used for the characters, and the interpolation of first 
degree or the like is used for the photographs. 

The following describes the variable magnification 
process using the above methods. 

In other words, in the present embodiment, the 
region segmentation section 3 detects possibilities of 
characters, photographs and mesh dots on a picture 
element so as to output the detected possibilities as the 
region segmentation data 3b. Then, calculation is made 
according to the equation (1) based upon the region 
segmentation data 3b. When the output picture element 
region segmentation data Xp takes a value representing 
the strongest possibility of characters, namely. Xp e 0, 
the equation (1) becomes as follows: 

and thus the nearest neighbor interpolation is used. 
Moreover, when the output picture element region seg- 
mentation data Xp takes a value representing the 
strongest possibility of photographs, namely, Xp « 7. the 
equation (1) becomes as follows: 

Dp = (PB-Da + PA •Db)/(PA + PB) 

and thus the interpolation of first degree is used. 

An output image obtained by the above computa- 
tion is shown in FIG. 13. In other words. FIG. 13 Is a 
graph which shows outputs in the case where when the 
read picture element shown in FIG. 10 is enlarged twice 
according to the region segmentation data 3b, two kinds 
of the variable scaling processes are used. Namdy, 
when the picture element is judged to be a character 
section, the nearest neight)or interpolation is used, and 
when the picture element is judged to t>e a photographic 
section, the interpolation of first degree is used. 

In this case, on the section which was judged to be 
the charader section in the region segmentation section 
3, namely, the left section, the edge is enhanced by 
using the nearest neighbor interpolation as the variable 
scaling method, and thus a dear innage is obtained. 
Moreover, in the region segmentation section 3. when 
the interpolation of first degree is used as the variable 
scaling method for the sedion which was judged to be a 
photographic section, namely, the right half, the tone 
produdion becomes satisfactory, arKJ thus a pseudo- 
contour is difficuttly caused. 

In such a manner, at the time of variable magnifica- 
tion process, referring to the region segmentation data 
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3b. the variable magnification process which is closer to 
the nearest neighbor interpolation is used on a section 
with the strong possibility of characters, and the variable 
magnification process which is closer to the interpola- 
tion of first degree is used on a section with the strong 
possibility of photographs. As a result, the density data 
which is scaling-processed can be made the most suit- 
able. 

As mentioned above, in the image processing 
method of the present embodiment which reads an 
image and divides the read image into blocks com- 
posed of a plurality of picture elements and performs a 
variable scaling process such as enlargement and 
decreasing of an image by making interpolation per pic- 
ture element, posstoilities of characters, photographs 
and mesh dots on each picture element on the image 
are detected by the region segmentation section 3. and 
interpolated picture element data per picture element 
are calculated by the variable scaling section 4 based 
upon the results of the detection so that scaling of the 
image is performed. 

In other words, in the conventional method of 
selecting the nearest neighbor interpolation or the inter- 
polation of first through third degrees, at the time of the 
enlarging process of the variatile scaling process, i.e.. 
enlargement, when the nearest neighbor interpolation is 
selected for the case where characters, photographs, 
etc. coexist in a document image, a pseudo-contour is 
occasionally caused on the photograph region, and 
when the interpolation of first through third degrees is 
selected, an edge of the characters becomes unclear. 

However, in accordance with the above method, the 
possibilities of characters, photographs and mesh dots 
on each picture element on an image are detected by 
the region segmentation section 3. and the interpolated 
picture element data per picture element are conputed 
by the variable scaling section 4 based upon the 
detected results so that scaling of the image is per- 
formed. 

Therefore, a section with the strong possibility of 4o 
characters is enlarged by the nearest neighbor interpo- 
lation or the like, and thus an edge of the enlarged ctiar- 
acters is prevented from becoming unclear. Meanwhile, 
a section with the strong possibility of photographs is 
enlarged by calculation which is closer to the interpola- 4s 
tion of first through third degrees, and thus a pseudo- 
contour of the enlarged photograph is prevented from 
being caused. 

As a result even if characters, photographs and 
mesh dots coexist on an image read by a scanner, a so 
deterioration in image quality can be prevented by scal- 
ing the image according to the characters, photographs 
or the mesh dots. 

In addition, in the image processing method of the 
present embodiment the region segmentation data rep- ss 
resenting the possibilities of characters, photographs 
and mesh dots detected by the region segmentation 
section 3 can take a value X which falls within a range of 
0 to N-1 (N is an integral number of not less than 2) (the 



smaOer the value X Is. the stronger the possibility of 
characters is. and the larger the value X is. the stronger 
the possibility of photographs or mesh dots). Moreover, 
when the region segmentation data of an interpolation 
5 picture element P whose density is should be deter- 
mined is represented by Xp, the density of the picture 
element A which is the closest to the picture element P 
Is represented by D^. the density of the picture element 
B which is the second closest to the picture element P is 
10 represented t>y Db. a distance between the picture ele- 
ments P and A is represented by PA, and a distance 
between the pcture elemerrts P and B is represented by 
PB, the variable magnrfication processing section 4 
computes the density Dp of the picture element P 
75 according to the aforementioned equation (1). 

In other words, sections of characters arKi sections 
of photographs and mesh dots mostly coexist on an 
actual image, and thus if the variable magnification 
method such as pattern matching is used for each pic- 
20 ture element of the above-mentioned image, there 
arises problems that a lot of memories are required for 
performing precise variable magnification process and 
that the processing time becomes longer. 

However, in accordance with the above method, the 
25 variable magnification processing section 4 performs 
the computation according to the equation (1). Then, in 
the equation (1). if the region segmentation data Xp tate 
a value 0 representing complete character, K = 0 and 
^P°P«- Namely, this means the variable scaling 
30 proc^ by means Of the nearest neighbor interpolation. 
Meanwhile, if the region segmentation data Xp 
takes a value N-1 representing a perfect photograph, for 
example, the density Dp t>ecomes as follows: 

Dp = (PB • + PA • Db) /(PA + PB) 

and thus Dp becomes a weighted linearly arithmetic 
means of the density Dg of the picture element A which 
is the closest to the picture element P and the density 
Dt> of the picture element B which is the second closest 
to the picture element P. Namely, this means the varia- 
ble scaling process by means of the interpolation of first 
degree. 

If the region segmentation data Xp takes a value 
between the possibilities of characters and photographs 
which fall within a range of more than 0% to less than 
100%, the density can be calculated based upon the 
weight according to the equation (1). 

Here, when the weighted linearly arithmetic means 
of Da and Db is represented by Dab. the aforementioned 
equation (1) can be changed as follows: 

Dp = (HXp /(N.1))} X + (Xp / (N-1)) X D^ 

As is clear from the above equation, when N is an 
integral number of not less than 3. the region segmenta- 
tion data Xp fulfills a relationship 0 < Xp < N-1 . 
namely, obtains a value between tiie possibilities of 
characters and photographs which falls within a range 
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Of more than 0% to less than 100%. Dp becomes a 
weighted linearly arithmetic mean of and Dg^y. Then, 
as Xp Is smaller. Dp takes a value which is closer to Da, 
and as Xp is larger. Dp takes a value which is closer to 

Therefore, even if characters and photographs or 
mesh dots coexist in an Image, during the variable scal- 
ing process, a simple sunoming and multiplying opera- 
tion represented t>y the equation (1) is perfonmed based 
upon the region segmentation data which represent the io 
possibilities of characters, photographs and mesh dots 
detected by the region segmentation means. More spe- 
cifically, the variable scaling method which is closer to 
the nearest neighbor interpolation Is used for the sec- 
tion with the strong possibility of characters P(p is is 
small), and the variable scaling method which is doser 
to the interpolation of first degree is used for the section 
with the strong possibility of photographs (Xp is large). 
As a result, more suitable density can be determined for 
the magnification varied data. Namely, since the density 20 
can be determined suitably for the characters, photo- 
graphs or mesh dots on each picture element by excel- 
lent software, the density can be determined quickly by 
a simple arrangement of hardware, and a deterioration 
in image quality can be prevented. 2S 

In the present embodiment the interpolation of first 
degree is used for determining density, but the method 
is not limited to this, and thus the interpolation of second 
or third degree can be used, for example. 

In addition, the above explanation refers to the 3o 
enlarging process of the variable magnif ication process, 
but the image processing method of the present embod- 
iment is not necessarily fimrled to this, and thus this 
process can be applicable to the case where decreas- 
ing process is performed. as 

For example, when an image in which characters 
and photographs coexist is read, the read image data 
are represented by a graph shown in FIG. 14. The left 
half of FIG. 14 shows a character region where the den- 
sity changes rapidly, and the right half shows a photo- 4o 
graphic region in which density changes continuously 
and comparatively mildly. 

When the nearest neighbor interpolation, in which 
density of a picture element existing in the nearest 
neighborhood of a target picture element of a reduced 4S 
image is used as the density of the target picture ele- 
ment, is used for the whole image at the time of reduc- 
ing the read image data to half, an output image shown 
in FIG. 15 is obtained. Namely, in the left half of the out- 
put image, i.e.. the character region, an edge is so 
enhanced comparatively, and thus the image becomes 
clear, but in the right half. i.e.. the photographic region, 
tone production is deteriorated, and thus a pseudo-con- 
tour is iiable-to be caused. 

In addition, when the read image data showvn in ss 
FIG, 14 are reduced to half by the interpolation of first 
degree which uses a weighted linearly arithmetic mean 
of the density of a picture element which exists in the 
nearest neighborhood of a target picture element of an 



output image and the density of a picture element which 
is the second closest to the target picture element as 
the density of the target picture element, an output 
image shown in FIG. 16 is obtained. Namely, in the right 
half of the output image, i.e., the photographic region, 
the tone production becomes satisfactory, and thus the 
pseudo-contour is hardly caused, but in the left half, i.e., 
the character region, an edge becomes unclear. 

Therefore, in the present embodiment, when the 
region segmentation data are judged to be the charac- 
ter section, the nearest neighbor interpolation is used, 
ard when the region segmentation data are judged to 
be the photographic section, the interpolation of first 
degree is used. A graphic of output image data obtained 
by the above method is shown in FIG. 17. In the left half 
of the FIG. 17, i.e., the region judged to be the character 
section, the edge is enhanced, and thus a clear image 
is obtained, and in the right half, i.e., the region judged 
to be the photographic section, the tone production 
becomes satisfactory, and thus a pseudo-contour ts 
hardly caused. 

In such a manner, the region segmentation data is 
referred to at the time of the reducing, and the variable 
magnification processing m^hod which is closer to the 
nearest neighbor interpolation is used in a section with 
the strong possibility of characters, and the variable 
magnification method which is closer to the interpola- 
tion of first degree is used in a section with the strong 
possibility of photographs. As a result, the density of the 
reduced data can be determined wore suitable. 

[EMBODIMENT 2] 

The following describes another embodiment of the 
present invention on reference to FIGS. 5. 10. 18 and 
19. Here, for convenience of explanation, those mem- 
bers that have the same arrangement arxj functions, 
and that are described in the aforementioned emt>odi- 
ment 1 are indicated by the same reference numerals 
and the desaiption thereof is omitted. 

In the image processing method of the aforemen- 
tioned embodiment if the possibility of characters is 
strong, the nnagnification is varied by the nearest neigh- 
bor interpolation. However, if the nearest neight3or inter- 
polation is used for an edge of characters, the edge is 
not enhanced suffidentiy 

Therefore, in the image processing method of the 
presertt embodiment, the nearest neighbor interpolation 
is not just used for an edge of characters, a method of 
multiplying density data obtained by the nearest neigh- 
bor interpolation by enlargement scale of the various 
scaling (i.e., lo enlarge by magnrfication") is adopted. 

An output image which was subject to the variable 
scaling process is shown in FIG. 18. 

In other words, in order to enlarge a read picture 
element shown in FIG. 10 twice, when tiie region seg- 
mentation data is referred to and are judged to be char- 
acter section, the nearest neighbor interpolation is 
used. Moreover, the variable scaling process is per- 
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formed on a section, which was judged to be a photo- 
graphic section, by the interpolation of first degree 
Therefore, the output image shown in FIG 13 is 
obtained. 

However, in the present embodiment when the s 
density of a picture element judged to be character sec- 
tion takes a value representing haH tone, rwmely. in the 
case of an edge of characters, the nearest neighbor 
interpolation is not just used, but the magnification of 
the density data obtained by the nearest neighbor inter- io 
polation is enlarged by magnification and the pulse 
width is corrected. 

A judgment can be made as to whether or not the 
density of the picture element takes a value represent- 
ing half tone by whether or not the density of the picture is 
element has a value whtah falls within a range of 0 to 
255. 

More specifically, as shown in FIG. 18, pulse widths 
(density of picture element) are corrected on sections of 
edges a and a' and edges b and of characters. First. 20 
a density, which was obtained by multiplying the density 
of the picture element obtained by the nearest neighbor 
interpolation by the magnification, is given to one pic- 
ture element on each side of the density slope where 
the density is high in two picture elements of the edges, ss 
namely, the picture element on the edge a* side and the 
picture element on the edge b side. 

Meanwhile, the dereity 0 is given to one picture ele- 
ment on each side of the density slope where the den- 
sity is low in two picture elements of the edges, namely 30 
the picture element on the edge a side and the picture 
element on the edge b' side. 

According to this con-ection of the pulse width (den- 
sity of a picture element), the density of the picture ele- 
ments on the inside in the two picture elements of the 35 
edges of characters is muWpned by the magnification, 
and the density of the picture elements on the outside is 
changed to 0. As a result, the edge of characters is 
enhanced more than the use of the nearest neighbor 
interpolation, and thus a clear image can be obtained. 4o 

The nearest neighbor interpolation is used in princi- 
ple for the section, which was judged to be the character 
section by the region segmentation section 3. namely 
the left section of the drawing (the section marked with 
"judged to be character section"), and the pulse widths 45 
on the edges of characters are corrected. As a result, 
the edges are enhanced more than the case of FIG. 13. 
and thus a clear image can be obtained. 

In addition, in the section, which was judged to be a 
photographic section by the region segmentation sec- so 
tion 3. namely the right half of the drawing (the section 
marked with "judged to be photographic section"), the 
tone production becomes satisfactory by adopting the 
interpolation of first degree, and a pseudo-contour is 
hardly caused. 

The pulse width modulator 16 outputs images for 
each picture element based upon the special data 4c 
including the picture element density slope data, and 
the scaling-processed y con-ected image data 5a. As 



mentioned in embodiment 1. when the special data 4c 
of 2 bits per picture element take OOB representing 
being centered, the pulse width modulator 16 generates 
continuous ON signals so as to be centered on the pic- 
ture element 

More specifically as shown in FIG. 5(c). the input 
image data is 80H. i.e.. has density of 128/256 and the 
input picture element density slope data are OOB repre- 
senting being centered, one-fourth (64^56) picture ele- 
ment from the front is turned off. and the next half of 
picture element is turned on. and the rest one-fourth of 
the picture element is turned off. In such a manner, the 
pulse width modulator 16 changes an ON data position 
of one picture element by adjusting an output of a laser. 

Therefore, the image data which were subject to the 
character edge process are r^resented by FIG. 19(a). 
for example. The small rectangles 6 represent respec- 
tively one output picture element, and numerical values 
of the rectangles 8 represent density of each picture 
element (256 tone). 

When the input picture element density slope data 
are OOB representing being centered, an image which is 
outputted based upon this image date is represented by 
FIG. 19(b). Namely when the input picture element 
density slope data are the OOB representing being cen- 
tered, a gap of an OFF signal is generated on an edge 
section. This is called as a ghost contour. 

Therefore, in the present embodiment, when the 
position of the ON signal in one picture element is cen- 
tered, in FIG. 19(b). the output control section 6 
changes the position of the ON signal in one picture ele- 
ment by modulating the pulse width of this picture ele- 
ment so as to prevent a ghost contour. 

In other words, picture elements on second through 
fourth rectangles on the leftmost file from the top 80H 
(i.e., 128) as sfwwn in FIG. 19(a). and the second file 
from the left is FFH fi.e., 255). As a result, the density 
slope on the picture elements is such that the density on 
the left side is low and the density on the right side is 
high. 

In addition, in FIG. 19(b). picture elements on the 
second thnjugh fourth rectangles on the rightmost file 
from the top are 80H as shown in FIG. 1 9(a). the second 
file from the right is FFH. As a result, densrty slope on 
these picture element is such that the densrty on the left 
side is high and the density on the right side is low. 

Therefore, according to this information, when the 
density slope of target picture elements is such that the 
density on the left side is low and the densrty on the right 
side is high, a picture element output is changed to data 
(10B) representing being shifted to the right and when 
the densrty slope of target picture elements is such that 
the density on the left side is high and the density on the 
right side is low, a picture element outpLit is changed to 
data (01 B) representing being shifted to the left 

In this manner, the output control section 6 changes 
the position of the ON signal in one picture elements by 
modulating the pulse width. As a result, as shown in 
FIQ. 19(c), a gap of the OFF signal is not produced on 
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an edge section of characters, thereby making it possi- 
ble to prevent the ghost contour. 

In accordance with the image processing method of 
the presait embodiment, picture element density slope 
data S of adjacent picture elements are given to each s 
picture element of an image. Meanwhile, in the case 
where the region segmentation data Xp of the picture 
element P is 0 (Xp=0), the density Dg of the picture ele- 
ment A which is the closest to the picture element P is 
half tone density and the picture element density slope io 
data Sp of the picture element P are poative. namely, 
data which represent ttmx Da < Dp < Db. the density Dp 
of the picture element P is converted into the value 
obtained by multiplying the half tone density and magni- 
fication together, and the density D^ of the picture ele- is 
ment A is set to 0. Moreover, when the density Da of the 
picture element A are half tone density and the picture 
element density slope data Sp of the picture element P 
are negative, namely, data which represent that Da > Dp 
> Db. the density D^ of the picture element A is con- 20 
verted into a value obtained by multiplying the telf tone 
density and magnification together, and the density Dp 
of the picture element P is set to 0. 

In other words, when the read image is character 
data and the picture elements on a reading position is 25 
an edge of the characters, the picture elements mostly 
have half tone density. Thereafter, when the variable 
scaling process is perfomned according to the equation 
(1). the picture elements having the half tone density 
continue, and thus the edge beconrtes unclear 30 

However, in accordance with the above method, 
when the density D^ of the picture element A which is 
the closest to the picture element P is half tone density 
and the picture element density slope data Sp of the pic- 
ture element P is positive, namely, Dg < Dp < D5. the as 
density Dp of the picture element P is converted into a 
value obtained by nrwltiplying the half tone density and 
magnification together, and the density Da of the picture 
element A is set to 0. Meanwhile, when the density Dg 
of the picture element A is half tone density and the pic- 40 
ture element density slope data Sp of the picture ele- 
ment P Is negative, namely Da > Dp > Db. the density Da 
of the picture element A is converted into a value 
obtained by multiplying the half tone density and magni- 
fication together, and the density Dp of the picture ele- 45 
ment P is set to 0. 

Therefore, when a pulse width of picture elements 
for outputting a laser is nruxlulated according to magnifi- 
cation, the density of edges of characters is further 
enhanced, and a clearer image can be obtained com- so 
pared with the case where the nearest neighbor interpo- 
lation is just used. 

In addition, in the image processing method of the 
present embodiment, a laser is outputted to a portion of 
picture elements where the density is high based upon ss 
the picture element density slope data on the picture 
elements to which a value obtained by multiplying the 
half tone density and magnification together was givoi. 
Namely, in the case where the read image is char- 



acter data and the picture element on the reading posi- 
tion is the edge of the image, when, for example. 8 
bits/picture element is processed and the picture ele- 
ment has the half tone density, the density of the picture 
element is 128. i.e., 80H. Therefore, an image output 
apparatus which modulates the pulse width on one pic- 
ture element so as to output a half-tone image, image 
data of 80H generates a half dot of one picture element 
in a center position of the picture elements. Then, in the 
case where the picture element where the half dot is 
generated in its center position is enlarged, a white pic- 
ture element of a half dot appears between the above 
picture element and the adjacent picture element, and 
the appearance of this white picture element is repeated 
for one picture element. Therefore, there arises a prob- 
lem that a ghost contour occurs. 

However, in accordance with the above method, a 
laser is outputted to a portion of picture elements where 
the density is high, based upon the picture element den- 
sity slope data on the picture elements to which a value 
obtained by multiplying the half tone density and magni- 
fication together was given. 

Therefore, when the edge of the character data is 
subject to the variable scaling process through the 
image processing method in claim 5. a ghost contour is 
prevented from being caused on the left or right of one 
picture element on the edge of the character data. 

[EMBODIMENTS] 

The following describes still another eoibodiment of 
the present invention in reference to FIGS. 12 and 24. 
Here, for convenience of explanation, those members 
that have the same arrangement and functions, and that 
are descrft>ed in the aforementioned enrbodiments 1 
and 2 are Indicated by the same reference numerals 
and the desatption thereof is omitted. 

As shown in FIG. 20. the digital copying machine of 
the present ennkxxjiment is arranged so that besides the 
anrangement shown in FIG. 1 . a second image process- 
ing section 15 is provided between the variable scaling 
section 4 and the output control section 6. A scaling- 
processed image can be transmitted from the second 
image processing section 15 to an asynchronizer 10 
such as a facsimile and a personal computer. 

In other words, as described in embodiment 1 . in 
the digital copying machine, the image input section 1 
reads a document image from a scanner, not shown, 
and converts the read image into the digital input Image 
data la. Next, the shading correcting/automatic expos- 
ing section 2 performs the shading correction and the 
automatic exposure for the input image data 1a. 

In addition, the region segmentation section 3 
refers to density, etc. of picture elements in the vicinity 
of a target picture element of image data 2a which were 
subject to the shading conrection and the automatic 
exposure so as to detect possibilities of characters, pho- 
tographs and mesh dots of the target picture element. 
Then, the region segmentation section 3 outputs image 



15 



BNSOOClOr< EP 07B5529A1 t > 



29 



EP 0 785 529 A1 



30 



data 3a and region segmentation data 3b. 

The variable scaling section 4 performs scaling 
process on the inputted image data 3a and the region 
segmentation data 3b, and refers to picture elements in 
the vicinity of the target picture element so as to deter- 
mine density slope of the target picture element Then, 
the variable scaling section 4 outputs scaling-processed 
inrage data 4a, scaling-processed region segmentation 
data 4b and special data 4c. 

At the time of the above variable scaling process, 
the equation (1) is used for the Image data 3a. and the 
nearest neighbor interpolation Is used for the region 
segmentation data 3b. Moreover, the special data 4c 
are composed of directing data, which show the front 
and rear of tiie line of the region segmentation data 3b 
and the end of the image data, and pcture element den- 
sity slope data S with respect to a picture element which 
IS currently subject to the process. Further, when the 
region segmentation data 3b are other than data which 
represent strong possibility of characters, the picture 
element density slope data S are outputted as "no 
slope"*. 

These outputs are outputted to the second image 
processing section 15. Then, they are transmitted to the 
asynchronizer 10 such as a facsimile and a personal 
computer, which is addrtionally provided, by a method, 
mentioned later. 

On the contrary, an image which is scaled by 
anotiier digital copying machine Is received by and 
inputted to the second image processing section 15 via 
the asynchronizer 10 such as facsimile. 

The scaling-processed image data 4a inputted into 
the second image processing section 15, the scaling- 
processed region segmentation data 4b and the special 
data 4c or the received and inputted variable magnifica- 
tion processed image are outputted as scaling-proc- 
essed r corrected image data 15a, scaling-processed 
region segmentation data 15b and specal data 15c. 
Then, image output is controfled by the output control 
section 6, and a laser is outputted based upon a signal 
of inputted output image data 6a by the Image output 
section 7. 

The following desaibed input and output of data 
from the variable scaling section 4 to the second image 
processing section 15. 

As shown in na 21 . the image data 3a stored in a 
line memory 17a and the region segmentation data 3b 
stored in a line memory 17b are inputted into the varia- 
ble magnification processing section 4. 

In addition, the output data from the variable magni- 
fication processing section 4 are the special data 4c 
stored in a line memory 18a. the magnification varied 
image data 4a stored in a line memory 18b and the 
magnification^aried region segmentation data 4b 
stored in a One memory I8c. 

When the scaling-processed image data 4a and the 
scaling-processed region segmentation data 4b are out- 
putted. directing data, which represent starting of data, 
the rear of each line and end of the image data, are 



added to the special data 4c. The information which rep- 
resents tiie starting of data, the rear of each line and the 
end of the image data, is given at the same time when 
the input image data are inputted from the image input 
5 section 1. 

More specifically, in order to represent the starting 
of the scaling-processed image data 4a and the scaling- 
processed region segmentation data 4b as shown in 
FiGS. 22(a) and 22(b), start directing data 20 are out- 

w putted as the directing data, which represent the start- 
ing of data, on output special data as indicated by a 
arcle in FIG. 22(b). Moreover, in order to represent ttie 
rears of the lines of the scaling-processed Image data 
4a and the scaling-processed region segmentation data 
IS Abas shown in FIQS. 23(a) and 23(b), rear directing 
data 21 are outputted as directing data, which represent 
the rears of the lines, on the output special data as indi- 
cated by a circle in FIG. 23(b). Further, in order to rep- 
resent the end of the whole image data of the 
so magnification varied image data 4a and the scaling- 
processed region segmentation data 4b as shown in 
FIGS. 24(a) and 24(b). image data end directing data 22 
are outputted as directing data, which represent the end 
of the whole image data, on the output special data as 
25 indicated by a circle in FIG. 24(b). 

In such a manner, directing data 20. 21 and 22. 
which respectively represent the front and rear of a line 
and tfie end of image data are given to the special data 
4c. and tiie scaling-processed image data 4a and the 
30 scaling-processed region segmentation data 4b are 
combined so as to be outputted. As a result, a line coun- 
ter or the like is not required even in the second image 
processing section 15. and output data can be gener- 
ated by a simple arrangement of hardware so as to be 
35 transmitted to the asynchronizer 1 0. 

As mentioned above, the image processing mettiod 
of the present entoodiment reads an image, divides tfie 
read image into blocks conposed of a plurality of picture 
elements and interpolate each picture element so as to 
^ variably scale the image. In accordance with this image 
processing method, posstoilities of characters, photo- 
graphs and mesh dots of each picture elemwrt of the 
image are detected by the region segmentation section 
3, and the scaling-processed image data 4a as interpo- 
45 lated picture element data of each picture element are 
computed by the variable scaling section 4 based upon 
the detected result so as to scale tiie image. Meanwhile, 
at the time of the computation of tiie scaling-processed 
image data 4a per picture element, the region segmen- 
50 tetion data 3b detected by the region segmentation sec- 
tion 3 are subject to the variable scaling process based 
upon the result detected by the region segmentation 
section 3 so as to be outputted as the scaling-proc- 
essed region segmentation data 4b togettier with the 
S5 scaling-pmcessed image data 4a. 

In the case where the scaling-processed Image 
data 4a is desired to be outputted to the asynchronizer 
10 such as a facsimile and a personal computer, for 
example, even if ttie region segmentation data 3b are 
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not subject to the variat}le scaling process, a number of 
picture elements does not agree with the scaling-proc- 
essed image data 4a. Therefore* there arises a problem 
that the region segmentation data 3b cannot be used. 

However, In the above method, also the region seg- 
mentation data 3b detected by the region segmentation 
section 3 are sut^ect to the variable scaling process 
fc>ased upon the result detected by the region segmenta- 
tion section 3 so as to be outputted as the scaling-proc- 
essed region segmentation data 4b together with the 
scaling-processed im^ge data 4a. 

Therefore, since a number of picture elements of 
magnification varied image data agrees with that of 
region segmentation data, even if these data are output- 
ted to an extemal asynchronous image input/output 
device such as a facsimile and a personal computer, 
region segmentation data can be used. 

In addition, since the possibilities of characters, 
photographs and mesh dots are given also to the scal- 
ing-processed region segmentation data 4b. they are 
combined together so that the scaling-processed image 
can be outputted to the asynchronizer 10. 

In addition, in the case where the scalir^-proc- 
essed image data 4a and the scaling-processed region 
segmentation data 4b are simultaneously outputted. the 
image processing method of the present emtxxiiment 
simultaneously outputs the direction data 20. 21 and 22. 
which represent the front arKl rear of each line of plural 
picture elements on output data, and the end of image 
data. 

In other words, in the case where the scaling-proc- 
essed image data 4a are outputted to the asynchronizer 
10 such as a facsimile and a personal computer, when 
the directing data, which represerrt the front and rear of 
the lines and the end of the image data, do not exist in 
the scaling-processed region segmentation data 4b. a 
number of picture elements for one line and a total 
number of lines should be transmitted to an external 
asynchronizer 10. Furtiiermore, even if the at>ove nunv 
bers are transmitted to the synchronizer 1 0, a line coun- 
ter should be always provided. 

However, in accordance with the above method, 
since the directing data 20, 21 and 22, wtvch represent 
the front and rear of each line of plural picture dements 
on the output data and the end of the image data, are 
simultaneously outputted. a number of picture elements 
of one line and a total number of lines do not have to be 
transmitted to an external asynchronizer 10. thereby 
making it possible to avoid instaQation of the line coun- 
ter. 

In the present embodment. the nearest neightx)r 
interpolation is used for the variable scaling process of 
the region segmentation data 3b. but the method is not 
limited to this, and thus the scaling process can be per- 
formed based upon the region segmentation data 3b 
like scaling of image data. 

The invention being thi^ described, it will be obvi- 
ous that the same may be varied in many ways. Such 
variations are not to be regarded as a departure from 



the spirit and scope of the invention, and all such modi- 
fications as would be obvious to one skilled in the art 
are intended to be included within the scope of the fol- 
lowing claims. 

5 

CJafms 

1. An image processing method for scaling an image, 
comprising the st^s of: 

10 

dividing an image into blocks composed of a 
plurality of picture elements so as to detect 
region segmentation data, which represent 
possibilities of characters, phcto^aphs and 

15 mesh dots in a block of a target picture ele- 

ment, t>y region segmentation means per tar- 
get picture element; and 
computing density of the target picture element 
on the output image by using corrputing means 

so according to an equation into which density of a 

plurality of adjacent picture elements in the 
vicinity of the target picture element are input- 
ted, 

wherein in said conputing step, a weight 
25 of the density of each adjacent picture element 

in the equation is adjusted according to the 
region segmentation data of the target picture 
element. 

30 2. The image processing method according to daim 1 , 
wherein in said computing step, the weight of the 
density of each adjacent picture element in the 
equation is adjusted so as to fall within a range of a 
weight obtained by the nearest neight>or interpola- 

35 tion to weights obtained by the interpolations of first 
through third degrees. 

3. The image processing method according to claim 1 , 
wherein in said confuting step, as the possibility of 

40 Characters represented t>y the region segmentation 
data is stronger, a weight of density of an adjacem 
picture elements which is the closest to the target 
picture element in the equation is set larger, and as 
the possibilities of photographs and mesh dots rep- 

45 resented in the region segmentation data are 
stronger, the weight is set smaller. 

4. The image processing method according to claim 1 . 
wherein: 

so 

in said detecting step, the region segmentation 
data are set so as to take a value X which faHs 
within the range of 0 to N-1 (N is an integral 
number not less than 2) and so that as the pos- 
55 sibility of characters is stronger, the region seg- 

mentation data takes a smaller value and as 
the possibilities of photographs and mesh dots 
are stronger, the region segmentation data 
takes a larger value. 
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5. 



in said computing step, density Dp of a picture 
element P which is a target picture element 
whose density should be determined, ts com- 
puted according to the following equation: 



Dp = {1-K)xD3 + KxDb 



0) 



(However, 
K = {Xp/(N-1))x{PA/(PA+PB))) 
where Xp is region segmentation data of the io 
picture element P, is density of a picture ele- 
ment A which Is the closest to the picture ele- 
ment P. Db is density of a picture element B 
which is the second closest to the picture ele- 
ment P. PA is a distance between the picture is 
element P and the picture element A. and PB is 
a distance between the picture element P and 
the picture element B. 



The image processing method according to claim 4. 
further comprising the steps of: 



20 



obtaining picture element density slope data 
which represent density slope of the picture 
element P with respect to picture elements zs 
which are adjacent to the picture element P 
after said computing step; and 
on the picture element P whose region seg- 
mentation data Xp is 0 and whose density Dp is 
half-tone density, and on the picture element A 3o 
which is the closest to the picture element P, 
when the picture element density slope data 
are data representing a relationship D^ < Dp < 
Db. converting the density Dp irrto a value 
obtained by multiplying the half-tone density 3s 
and magnification of the various scaling 
together and setting the density Da to 0, 
whereas when the picture element density 
slope data are data representing a relationship 
Db < Dp < Da. converting the density Da into a 40 
value obtained by multiplying the half-tone den- 
sity and magnification together and setting the 
density Dp to 0. 

The image processing method according to daim 5. 4S 
further comprising: 

the step of outputting a laser according to den- 
sity data with respect to the picture element P 
after said converting step. so 

wherein in said laser outputting step, a 
laser is outputted to the picture element P. to 
which the value obtained by multiplyirig the 
half-tone density and the magnification 
together is given, in a position which is shifted ss 
to a side of the picture element where the den- 
sity is higher based upon the picture element 
density slope data of the picture element P 



7. The image processing method according to claim 1 , 
wherein in said computing step, when the data of 
each target picture element is computed by said 
computing means, magnification of the region seg- 
mentation data detected by said region segmenta- 
tion means is scaled so as to be outputted together 
with the scaled image data. 

8. The image processing method according to claim 7. 
wherein the variable magnification process for the 
region segmentation data in said computing step is 
performed by using the nearest neighbor interpola- 
tion. 

9. The image processing method according to claim 7, 
wherein the variable magnification process for the 
region segmentation data in said computing step is 
performed based upon the result detected by said 
region segmentation means. 

1 0. The image processing method according to claim 7. 
further comprising: 

the step of simultaneously outputting the 
scaled image data and the scaled region seg- 
mentation data after the computing st^, 

wherein in said outputting step, directing 
data, which represent a front and a rear of each 
line of plural picture elements on the output 
data and end of the image data, together wHh 
the scaled image data and the scaled region 
segmentation data. 

11 . The image processing method according to claim 1 . 
further comprising : 

the step of obtaining picture element density 
slope data which represent density slope of the 
target picture element with respect to picture 
elements which are adjacent to the target pic- 
ture element after said conputing step; and 
the step of outputting a laser according to the 
density data of each picture element. 

wherein in said laser outputting step, a 
laser is outputted to the target picture element, 
to which the value obtained by muftipl^ng the 
half-tone density and the magnification of the 
various scaling together is given, in a position 
which is shifted to a side of the picture element 
where the density is higher based upon the pic- 
ture element density slope data of the target 
picture element. 

1 2. The image processing method according to claim 1 . 
further comprising the step of reading a document 
image and converting the read image into image 
data so as to input the image data into said region 
segmentation means before said detecting step. 
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13. The image processing method according to claim 1 , 
wherein: 



said image processing method is a method of 
enlarging an image composed of a plurality of s 
picture elements by inserting interpolated pic- 
ture elements respectively between the picture 
elements, 

the target picture element of which the density 
computed in said computing step Is the interpo- io 
lated picture element 

14. The image processing method according to daim 
13, further comprising the step of reading a docu- 
ment image and converting the read image into 75 
image data so as to input the image data into said 
region segmentation means before said detecting 
step. 

15. An image processing apparatus for scaling inputted 20 
image so as to output the magnification varied 
image, comprising: 

region segmentation means for dividing the 
input image into blocks composed of a plurality 2s 
of picture elements and detecting region seg- 
mentation data, which represent possibilities of 
characters, photographs and mesh dots of a 
block of a target picture dement per target pic- 
ture element; and 30 
computing means for computing density of the 
target picture element on the output image by 
using an equation in which density of a plursdity 
of adjacent picture elements in the vicinity of 
the target picture element is inputted, 3S 

wherein said computing means adjusts 
a weight of the density of each adjacent picture 
element in the equation according to the region 
segmentation data of the target picture ele- 
ment 40 
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